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Approximation  concepts  are  an  effective  approach  for  alleviating  some  of  the 
problems  associated  with  the  direct  use  of  modern  computerized  analysis  techniques  in  an 
optimization  environment.  Response  surface  approximations  shift  the  computational 
burden  from  the  optimization  problem  to  the  problem  of  constructing  the  approximations. 
Additionally,  response  surface  approximations  filter  out  the  noise  inherent  to  most 
numerical  analysis  procedures,  by  providing  smooth  approximate  response  surface 
functions;  and  simplify  the  integration  of  analysis  and  optimization  codes.  Unfortunately 
response  surface  techniques  suffer  from  the  so-called  “curse  of  dimensionality”  and 
become  less  and  less  attractive  as  the  number  of  design  variables  increases.  This  work 
investigates  the  use  of  response  surface  (RS)  approximations  in  computationally 
expensive  structural  optimization  problems.  Two  methods  are  proposed  to  extend  the 
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applicability  of  response  surface  techniques  to  problems  that  deal  with  more  than  a few 
design  variables. 

The  first  approach  is  to  alleviate  computational  costs  by  taking  advantage  of  the 
simple  models  that  were  in  use  a generation  ago  when  computers  were  much  less 
powerful.  These  models,  termed  here  as  low-fidelity  models,  are  less  accurate.  However, 
they  can  be  combined  with  more  accurate  but  expensive  high-fidelity  models,  through  the 
creation  of  correction  response  surfaces,  to  provide  a good  combination  of  high  accuracy 
and  low  computational  cost.  Correction  response  surface  approximations  are  used  to 
solve  two  structural  weight  optimization  problems  efficiently:  the  design  of  the  top  cover 
panel  of  a blended-wing-body  airplane  and  the  design  of  a wing  composite  blade- 
stiffened  panel  subject  to  crack-propagation  constraints. 

The  second  approach  uses  response  surface  approximations  that  incoiporate  both 
function  and  derivative  data.  In  computer  simulations,  derivatives  often  are  available, 
frequently  at  a low  computational  cost.  In  theory  the  inclusion  of  derivatives  data  would 
allow  response  surface  approximations  to  be  applied  to  a problem  with  many  design 
variables.  However,  derivatives  are  usually  less  accurate  than  function  evaluations.  The 
present  study  attempts  to  identify  the  trade-offs  of  using  derivatives  in  constructing 
response  surfaces  through  simple  example. 
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CHAPTER  1 
INTRODUCTION 

Designing  an  engineering  system  is  often  a challenging  task.  Deriving  a suitable 
mathematical  formulation  that  accurately  describes  the  behavior  of  the  engineering 
system  can  be  even  more  challenging.  When  this  is  at  all  possible,  finding  a solution  to 
the  mathematical  formulation  of  the  system  might  be  even  more  difficult. 

In  many  instances,  however,  it  is  possible  to  modify  the  original  problem  into  a 
slightly  different  approximate  one  and  then  to  solve  exactly  the  approximate  problem. 
The  solution  obtained  may  be  viewed  as  an  approximate  solution  to  the  original  design 
problem.  In  many  other  instances  it  may  be  possible  to  solve  the  mathematical 
formulation  of  the  original  design  problem  using  an  approximation  technique.  The 
solution  is  once  again  viewed  as  an  approximate  solution  to  the  original  design  problem. 
Of  course,  for  a large  variety  of  problems  it  is  possible  to  solve  the  approximate  problem 
in  an  approximate  way  and  still  obtain  a satisfactory  solution  to  the  original  design 
problem. 

In  this  study,  particular  attention  is  paid  to  one  approximation  method  called 
response  surface  (RS)  technique.  Response  surface  methodology  approximates  the 
response  of  a system  as  a function  of  some  variables.  The  approximation  is  obtained  by 
fitting  the  system  response  for  a number  of  selected  combinations  of  the  control  variables 
(design  points).  Response  surface  approximations  have  been  shown  to  be  accurate  for  a 
small  number  of  parameters.  However  as  the  number  of  parameters  increases,  traditional 
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response  surface  techniques  require  an  increasing  number  of  evaluations  to  achieve 
acceptable  accuracy.  This  makes  them  computationally  intractable. 

The  study  focuses  on  methods  that  can  be  applied  to  response  surface  techniques 
to  design  optimization  problems  with  more  than  a few  design  variables.  In  particular  the 
two  approaches  that  were  studied  are  correction  response  surfaces  (CRS)  and  derivative 
based  RS.  The  objective  is  to  demonstrate  the  potential  of  these  two  techniques. 

Chapter  2 provides  a literature  review  on  the  state  of  the  art  in  approximation 
methods  applied  to  optimization  problems.  It  briefly  introduces  the  most  popular 
approximation  methods,  dividing  them  into  local,  mid-range,  and  global  approximations. 
Particular  attention  is  paid  to  global  methods  in  the  form  of  RS.  The  section  on  response 
surface  methodology,  being  the  main  focus  of  this  work,  is  discussed  in  greater  detail. 
The  mathematical  and  statistical  bases  of  response  surface  methodology  are  summarized 
briefly  in  Chapter  3.  Chapter  4 applies  of  correction  response  surface  to  the  optimization 
of  a hat-stiffened  panel  made  of  composite  materials.  Chapter  5 applies  correction 
response  surface  methodology  to  design  optimization  against  crack  propagation  of  a 
bladed  stiffened  panel,  also  made  of  composite  materials.  Chapter  6 deals  with  response 
surface  based  on  both  function  and  gradient  data.  The  approach  is  applied  to  numerical 
problem  where  control  on  the  noise  and  modeling  errors  could  be  enforced.  Concluding 
remarks  and  proposed  future  research  are  provided  in  Chapter  7. 


CHAPTER  2 
LITERATURE  REVIEW 


Approximation  concepts  in  structural  optimization  were  introduced  in  the 
mid-seventies  by  Schmit  and  Miura,  (1976)  and  by  Schmit  and  Farshi,  (1977).  They 
showed  that  applying  non  linear  programming  methods  to  large  structural  design 
problems  were  cost  effective,  provided  that  suitable  approximations  concepts  were 
introduced.  Several  other  publications  on  approximation  concepts  soon  followed  as  the 
approach  attracted  the  interest  of  many  other  researchers. 


Function  Approximation  Concepts 

The  use  of  approximation  techniques  to  solve  an  optimization  problem  allows  a 
designer  to  minimize  the  computational  effort  needed  to  find  a satisfactory  solution  and 
to  integrate  different  simulation  codes  if  more  then  one  code  is  needed  during  the  design 
process.  The  selection  of  appropriate  approximating  functions  to  replace  the  real 
functions  that  describe  an  optimization  problem  is  essential  to  ensure  that  the 
optimization  procedure  is  solved  correctly  and  efficiently. 

Barthelemy  and  Haftka  (1993)  distinguished  approximation  methods  based  on  the 
size  of  the  domain  of  their  validity,  as  local,  global,  and  mid-range  approximations. 

In  general,  an  optimization  problem  can  be  formulated  as  the  minimization  of  an 
objective  function  F(x ) 


F(x ) — » min 
Gj  (x) < 0 
Hk  (x)  = 0 

/ / / 14 
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where  the  vector  x e 91 7 in  Eq.  (2.1)  defines  a point  in  the  design  space  Q whose 


elements  are  the  design  variables.  G (x)  and  H k (x)  are,  respectively,  the  inequality  and 


equality  constraints  that,  together  with  the  lower  and  upper  bounds  of  the  design  variables 
x,  and  x," , define  the  feasible  design  domain.  These  lower  and  upper  bounds  of  the 

design  variables  also  are  called  side  constraints. 

Approximation  techniques  replace  the  problem  formulated  in  Eq.  (2.1)  by  a series 
of  approximate  problems.  The  solution  of  each  approximate  problem  is  called  a cycle.  At 
each  cycle  the  original  problem  in  Eq.  (2.1)  is  replaced  by 

Fp(x ) — > min 


Gp  (x)  < 0 j = 


Hp  (x)  = 0 k = \,...,K 
xl’p  < x,  < x'“'p  i = 1 ,...,/ 


(2.2) 


where  p is  the  cycle  number,  and  Fp (x) , Gp(x)  , and  Hp (x)  are  approximate 

expressions  for  the  objective  function,  inequality  constraints,  and  equality  constraints, 
respectively,  and  the  side  constraints  satisfy  the  condition 

x\'p  > x[  and  xtpp  < x"  (2.3) 

The  lower  and  upper  bounds  of  the  design  variables  change  from  cycle  to  cycle  in 
order  to  limit  the  search  for  the  solution  to  a region  where  the  approximations  are  valid. 
The  number  of  cycles  needed  to  reach  an  acceptable  design  may  vary  from  one,  in  the 
case  of  very  accurate  global  approximations;  to  thousands,  in  the  case  of  local 
approximations  that  are  valid  only  in  small  regions  of  the  design  domain. 
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Problems  in  Engineering  Optimization  based  on  Computer  Simulations 

Numerical  simulations  enable  engineers  to  perform  detailed  analyses  of  complex 

systems.  However,  the  use  of  such  simulation  tools  in  design  optimization  is  associated 
with  several  problems.  A numerical  simulation  provides  a designer  with  only  a single 
point  in  the  design  space  rather  than  an  overall  view  of  the  system  behavior  within  the 
design  space.  This  is  in  contrast  with  the  pre-computer  era,  when  algebraic  expressions 
and  design  charts  furnished  a global  view  of  the  system  behavior  in  the  design  space.  The 
algebraic  expressions  and  design  charts  allowed  the  designer  to  spot  promising  design 
regions  and  structural  configurations.  However  analysis  methods  available  in  the  pre- 
computer era  were  not  accurate  enough  for  the  design  of  complex  structures.  It  would  be 
beneficial  if  it  were  possible  to  generate  a global  view  of  the  design  space  by  producing 
algebraic  expressions  based  on  accurate  numerical  simulations. 

A second  problem  with  computer  simulations  is  that  human  errors  and  numerical 
noise  usually  are  present  when  dealing  with  large,  complex,  and  cumbersome  simulation 
software  such  as  finite  element  codes.  Numerical  noise  can  be  caused  by  mesh  generator 
procedures  that  update  the  element  mesh  as  the  design  changes  in  order  to  maintain 
required  accuracy.  Other  sources  of  noise  include  less  than  fully  converged  iterative 
procedures  and  round-off  errors.  Because  the  number  of  elements  and  grid  points  in  a 
finite  element  (FE)  model  is  discrete  by  nature,  finite  jumps  in  the  evaluated  response  can 
be  introduced  when  an  infinitesimal  change  in  the  FE  model  leads  to  adaptive  meshing 
that  changes  the  number  of  nodes  and  elements  or  the  topology  of  the  mesh.  Human  and 
numerical  errors  of  considerable  magnitude  are  easily  located  and  corrected,  so  normally 
the  remaining  errors  have  little  influence  on  the  accuracy  of  the  results.  However,  the 
remaining  numerical  noise  may  cause  non-smooth  behavior  of  the  response,  trapping  the 
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optimizer  in  spurious  local  optima  and  making  it  difficult  to  find  the  best  design  ( Giunta 
et  al.,  1994). 

When  analyzing  a complex  structure  using  a finite  element  code,  a source  of  noise 
is  introduced  implicitly  in  the  discretization  procedure.  Van  Keulen  et  al.  (1997) 
performed  an  extensive  study  of  the  influence  of  numerical  noise  on  the  optimal  solution 
of  several  structural  design  problems  for  different  discretization  densities.  They  showed 
that  high  levels  of  noise  can  be  tolerated  at  the  early  stages  of  optimization,  but  as  the 
optimization  progresses  and  more  information  about  the  behavior  of  the  response  is 
gathered,  the  level  of  noise  that  can  be  tolerated  is  reduced. 

Optimization  is  a procedure  that  searches  through  the  design  domain  for  the 
optimum  design.  During  this  search,  the  optimization  algorithm  continuously  calls  the 
analysis  code  and,  based  on  the  data  obtained,  it  decides  on  the  next  design  point  to 
evaluate.  This  requires  an  interface  between  the  analysis  and  optimization  programs. 
Some  general-purpose  finite  element  programs,  like  NASTRAN,  come  equipped  with  an 
optimization  algorithm  and  an  interface  between  optimization  and  analysis.  However, 
often  the  optimization  code  and  the  analysis  code  are  separate,  or  one  might  want  to  use 
an  optimization  algorithm  different  from  the  one  furnished  with  the  analysis  code.  In 
these  situations  the  user  must  write  an  interface  between  the  two  codes,  possibly  a 
cumbersome  and  frustrating  task. 

Finally,  some  optimization  problems  such  as  global  optimizations  or  stochastic 
optimization  require  a large  number  of  analyses  and,  except  for  simple  problems;  such  a 
large  number  of  analyses  are  computationally  impractical. 

As  we  show,  the  use  of  approximations,  especially  global  approximations,  helps 
alleviate  these  four  problems.  Approximations  can  provide  the  designer  with  a global 
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view  of  the  design  space,  filter  out  numerical  noise,  help  in  interfacing  analysis  and 
optimization  software,  and  permit  us  to  perform  global  or  stochastic  optimization. 

Local  Approximations 

Local  approximations  are  probably  the  most  commonly  applied  approximations  in 
structural  optimization.  Generally,  they  are  based  on  the  function  value  and  sensitivities 
at  one  point  of  the  design  domain.  Much  literature  has  been  written  over  the  years  on 
these  types  of  approximations  (e.g.,  Barthelemy  and  Haftka,  1993).  As  their  name 
suggests,  local  approximations  are  only  accurate  in  the  vicinity  of  the  point  at  which  they 
are  generated. 

A distinction  should  be  made  between  local  function  approximation  and  local 
problem  approximation.  Local  function  approximations  are  variations  on  Taylor  series 
expansions,  while  local  problem  approximations  try  to  reduce  the  size  of  the  active 
constraint  set  or  the  design  variable  set. 

One  of  the  first  robust  optimization  algorithms  was  the  simplex  algorithm 
(. Dantzig , 1963)  for  the  solution  of  linear  optimization  (known  as  linear  programming) 
problems.  It  was  only  natural  that  the  first  approximation  methods  developed  used  linear 
programming.  Nonlinear  optimization  problems  were  linearized  about  a point  at  each 
optimizaton  cycle  and  the  simplex  method  was  used  to  find  the  optimum  in  the  small 
neighborhood.  The  cycle  was  repeated  until  convergence  leading  to  the  so-called 
sequential  linear  programming  (SLP)  method. 


For  a linear  approximation,  the  functions  Fp(x ) , G'"(x) , and  ///’ (x ) in  Eq.  (2.2) 


are  linear  Taylor  expansions.  For  example 
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where  x0  is  the  design  point  about  which  linearization  is  performed. 

It  was  realized  soon  that  for  some  problems  more  accurate  approximations  could 
be  obtained  for  the  same  computational  cost.  The  first  to  come  was  probably  the 
reciprocal  approximation,  where  the  linear  approximation  of  F(x)  is  replaced  by  a linear 

approximation  in  F(y)  where 


y,=-  (2.4) 

X, 

This  substitution  usually  helps  because  in  many  structural  design  problems  the 
design  variables  are  truss  cross-sectional  areas,  moment  of  inertia  of  beams  and 
thicknesses  of  plates.  Displacements  and  stresses  of  statically  determinate  structures  are 
linear  in  the  reciprocals  of  these  design  variables,  and  for  statically  indeterminate 
structures  approximate  linearity  often  holds.  One  problem  with  the  reciprocal 
approximation  is  that  it  becomes  unbounded  when  the  design  variables  approach  zero. 
One  way  to  avoid  the  problem  was  to  make  use  of  the  modified  reciprocal 
approximations  ( Haflka  and  Shore,  1979).  Another  first-order  approximation  method, 
called  the  conservative  approximation,  was  introduced  by  ( Starnes  and  Haftka,  1 979). 
This  method  is  a hybrid  form  of  the  linear  and  reciprocal  approximations  methods  that  is 
more  conservative  than  either.  The  original  or  reciprocal  approximations  are  used 
according  to  the  sign  of  the  product. 


x 


o i 


( d, F(x)^ 

Sx, 


(2.5) 


o 


One  advantage  of  the  conservative  approximation  is  that  it  renders  the  problem 


convex  and  therefore  is  guaranteed  to  have  only  one  global  minimum.  Several 
researchers  proposed  different  approximation  methods  that  had  the  desirable  property  of 
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convexity;  the  most  successful  one  may  be  the  method  of  moving  of  asymptotes 
introduced  by  Svanherg  (1987).  However  the  reciprocal  and  the  conservative  methods  as 
well  as  other  convex  approximation  destroy  the  linearity  of  the  problem,  thereby 
rendering  impossible  the  use  of  SLP. 

Higher  derivatives  can  be  used  in  local  approximations,  but  the  increase  in 
accuracy  often  is  not  worth  the  computational  cost  of  the  additional  derivatives. 

Quadratic  approximations  primarily  have  been  used  in  solving  eigenvalue  problems. 
However,  Murthy  and  Haftka  (1988)  showed  that  for  almost  the  same  computational  cost 
it  is  possible  to  obtain  a third  order  approximation  by  using  a linear  approximation  to  the 
eigenvectors  in  the  Rayleight  quotient.  Often  the  high  cost  of  obtaining  second 
derivatives  dictates  a compromise  leading  to  the  use  of  only  the  diagonal  elements  of  the 
second  derivatives  matrix  ( Fleury , 1 989). 

Using  approximations,  it  is  not  necessary  to  approximate  all  response  quantities 
considered  in  the  optimization  process.  Using  the  process  of  local  problem  approximation 
we  can  ignore  constraints  that  are  not  critical  or  near  critical  for  the  current  cycle  in  the 
optimization  process.  This  approach,  referred  to  as  constraint  deletion  or  constraint 
screening,  was  first  introduced  by  Hang  and  Aurora  (1979).  It  should  be  noted  that 
constraint  deletion  is  used  only  to  reduce  the  computational  effort  but  that  it  does  not 
affect  the  final  optimum  achieved.  For  a more  complete  overview  of  the  process  see 
Vanderplaats  (1999).  Moreover  when  dealing  with  complex  FE  models,  many  elements 
in  a small  region  of  the  structure  have  approximately  the  same  stress.  It  is  possible  to  use 
the  most  critical  element  to  represent  the  response  in  that  region.  This  approach,  known 
as  regionalization,  was  introduced  by  Schmit  and  Miura  (1976).  The  result  of  these  two 
simple  techniques  is  that  while  the  structure  may  have  several  thousand  constraints,  the 
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sensitivity  of  only  a small  fraction  of  them  has  to  be  evaluated,  leading  to  considerable 
savings  in  computational  cost. 

Local  approximations  have  been  successfully  applied  to  numerous  problems  but 
they  have  some  drawbacks.  First,  they  do  not  provide  the  designer  with  a global  view  of 
the  design  space.  Second,  local  approximations  provide  only  limited  protection  against 
numerical  noise.  Finally,  local  approximations  are  not  well  suited  for  parallel  computing 
without  non-trivial  software  changes. 

Mid-Range  Approximations 

Mid-range  approximations  are  valid  in  a region  of  the  design  space  that  is 
considerably  larger  than  local  approximations  but  smaller  than  the  entire  design  space. 
Unlike  local  approximations,  data  from  more  than  one  design  point  are  used  to  construct 
mid-range  approximations.  In  local  methods,  a new  approximation  is  constructed  at  a 
new  design  point,  discarding  previous  analyses.  Some  mid-range  approximation  methods 
work  in  a similar  way,  but  now  at  each  cycle  more  than  one  point  is  used  to  obtain  an 
approximation  valid  in  a region  of  the  design  domain.  Mid-range  methods  use  data  from 
previously  generated  design  points  to  enhance  the  accuracy  of  the  approximations.  In  the 
literature  two  types  of  mid-range  approximation  methods  that  use  the  previously 
generated  data  can  be  distinguished.  The  first  type  uses  data  from  individual  points  along 
the  optimization  path  in  the  design  space.  Such  approximations  are  called  single-point 
path  (SSP)  approximations.  Approximations  derived  from  data  computed  in  clusters  of 
design  points  along  the  optimization  path  are  called  multiple  point  path  (MPP) 
approximations. 

In  SPP  approximation,  the  design  points  used  for  the  approximations  are  the 
solutions  of  the  previous  optimization  cycles.  Vanderplaats  (1979)  developed  Taylor 
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series  expansions  about  the  current  design  point  in  optimizing  airfoils  using  function 
values  from  all  previous  design  points.  Two  and  three-point  approximations  were 
proposed  by  Haftka  et  al.  (1987).  The  authors  generated  approximations  based  on  the 
projection  of  a design  point  on  the  line  connecting  two  points,  or  onto  the  plane 
connecting  three  points,  approximating  the  functions  by  Hermite  polynomials  at  the 
projection  point.  While  tests  showed  improvement  over  local  linear  approximations, 
reciprocal  methods  still  gave  better  results.  In  addition  the  results  indicated  that  the 
approximations  were  good  for  interpolations,  while  the  improvements  in  accuracy  were 
marginal  for  extrapolations.  Fadel  et  al.  (1990)  presented  a two-point  exponential 

approximation  in  terms  of  the  intervening  variables  yi  = xp  , i = 1 , ,1.  The  exponent  p, 

for  each  design  variable  was  determined  by  matching  the  derivatives  of  the  approximate 
function  with  the  previous  data  point  gradients. 

Grcmdhi  and  co-workers  (e.g.,  Wang  and  Grandhi,  1994,  Wang  and  Grandhi. 

1 995  and  Xu  and  Grandhi,  1 999)  constructed  a family  of  algorithms  based  on  intervening 
variables  called  TANA,  where  the  intervening  variables  were  defined  as  in  Fadel  el  al.  as 

yt  = xy  for  i = 1 , ,1.  For  example,  the  TANA3  algorithm  of  Xu  and  Grandhi  (1999) 

uses  a second-order  Taylor  series  expansion  in  terms  of  the  intervening  variables,  with  a 
diagonal  Hessian  matrix.  The  exponent  of  each  design  variable  and  the  diagonal  second- 
order  terms  are  evaluated  by  matching  the  derivatives  and  the  function  value  of  the 
approximate  function  with  the  previous  function  and  gradient  data.  In  other  words,  unlike 
the  approach  followed  by  Fadel  et  al.,  TANA3  matches  the  function  value  and  the 
gradient  of  a previous  design  point  rather  than  only  the  gradient.  Rodriguez  et  al.  ( 1 998) 
implemented  an  algorithm  that  is  based  on  an  approximation  of  the  Hessian  matrix.  The 
authors  approximated  the  Hessian  matrix  for  their  problem  by  matching  the  function  and 
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gradient  values  of  the  response  surface  approximation  with  the  values  obtained  from 
numerical  experiments  in  a region  of  the  design  space  at  additional  design  points  around 
the  current  one. 

An  improvement  to  local  methods,  known  as  the  accumulated  function 
approximation,  was  developed  by  Rasmussen  (1996).  Starting  from  an  arbitrary  design 
point,  a local  approximation  is  constructed  and  a possible  optimum  point  is  found  in  a 
design  space  larger  than  the  space  of  validity  of  the  local  approximation.  Based  on  the 
function  value  and  possibly  the  gradients  at  the  starting  point  and  newly  obtained  point,  a 
new  approximation  is  found  for  the  design  space.  The  process  allows  including 
information  from  more  points  as  they  become  available.  A weighting  procedure  is  used 
so  that  points  closer  to  the  last  point  have  more  weight  in  defining  the  approximation. 
Rasmussen  (1996)  showed  that  even  a considerable  level  of  noise  in  the  sensitivities  can 
be  tolerated  by  this  method.  No  noise  in  the  function  values  was  considered. 

The  MPP  methods  generate  one  or  more  extra  design  points  around  each  solution  of  an 
optimization  cycle,  enabling  the  approximations  to  use  these  additional  points.  The 
additional  points  can  be  chosen  according  to  design  of  experiments  discussed  in  Khuri 
and  Cornell  (1996).  Free  et  al.  (1987)  used  a central  composite  design  (CCD)  scheme  to 
select  design  points  used  to  construct  low-order  polynomials  in  a region  of  the  design 
domain.  They  reported  competitive  efficiency  with  conventional  optimization  algorithms 
when  no  noise  is  present.  When  noise  is  present,  the  method  was  more  efficient  than 
conventional  algorithms.  A general  MMP  concept  was  formulated  by  Toropov  et  al. 

(1999)  in  which  any  function  F (x)  in  Eq.  (2.2)  can  be  used  as  an  approximation 
function.  These  models  can  be  fitted  to  a function  F (x)  by  means  of  weighted  least- 


squares  method  as 
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min  Z(w-  (F(X,)~F(X  ))2  + E wjj 
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Kdx,  d x,  Jx 


(2.7) 


where  w,  eontains  the  weight  factors  expressing  the  relative  contribution  of  each  design 
point.  Note  that  in  this  approach  the  use  of  first-order  derivatives  is  not  mandatory.  The 
procedure  to  construct  function  approximation  is  comparable  to  global  methods.  The 
difference  is  that  the  region  where  the  approximation  is  used,  bounded  by  move  limits, 
moves  through  design  space  as  the  optimization  progresses.  Van  Keulen  et  al.  (1995) 
used  the  multi-point-path-approximation  mid-range  methods  successfully  to  solve  of  a 
shape  optimization  problem.  A similar  approximation  concept  was  applied  successfully 


by  Etman  et  al.  (1996). 


Global  Approximations 

For  some  problems,  it  could  be  profitable  to  construct  a global  approximation 
instead  of  a local  or  mid-range  one.  One  of  the  most  commonly  used  methods  for  global 
approximations  is  the  response  surface  (RS)  technique,  originally  developed  for  the 
planning  and  analysis  of  physical  experiments  (e.g..  Box  and  Draper,  1987).  Because  of 
similarities  between  physical  experiments  and  computer  simulations,  response  surface 
techniques  have  been  used  to  build  explicit  approximations  to  structural  response  from 
numerical  experiments  in  the  form  of  finite  element  simulations. 


Response  Surface  Techniques 

When  dealing  with  complex  problems  the  true  response  or  performance  measures 
as  a function  of  the  design  variables  usually  is  not  known  explicitly.  In  these  situations 
design  optimization  based  on  experiments  may  be  the  only  tool  a designer  has. 
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For  example,  a farmer  may  want  to  maximize  the  yield  per  acre  of  crops  by 
varying  the  quantity  of  two  fertilizers.  The  true  value  of  the  yield  as  a function  of  the  two 
fertilizers  is  not  known,  and  it  may  not  be  derived  explicitly  . The  farmer  in  this  case  can 
perform  several  experiments  on  different  plots  using  different  quantities  of  the  two 
fertilizers  and  can  record  the  yield  for  each  combination  of  the  fertilizers.  Once  the  data 
from  the  experiments  has  been  gathered,  an  analytical  expression  relating  the  two  design 
variables  (fertilizers)  and  the  response  (yield  per  acre)  can  be  fitted  to  the  data  to  generate 
a response  surface  as  a model  of  yield.  Most  commonly,  linear  or  quadratic  polynomials 
are  used.  However,  noise  in  the  data  may  be  introduced  due  to  different  skills  of  workers 
tending  the  plots  or  inaccuracies  in  measuring  fertilizer  amounts  or  yields.  The  farmer 
would  probably  like  to  conduct  a large  number  of  experiments  to  reduce  the  effect  of  the 
noise,  but  at  the  same  time  he  or  she  would  like  to  keep  the  number  of  experiments  small 
to  minimize  cost. 

Response  surface  techniques  help  the  farmer  to  select  combinations  of  the  two 
fertilizers  to  try  in  order  to  gain  the  maximum  information  from  the  minimum  number  of 
experiments.  In  addition,  after  the  analytical  expression  of  the  response  is  fitted  (a 
process  called  regression)  response  surface  methodology  helps  the  farmer  establish 
whether  the  relationship  found  is  statistically  meaningful.  Can  the  farmer  trust  the 
relationship  found  to  predict  the  yield  of  crops  for  all  the  possible  combinations  of  the 
fertilizers?  What  is  the  expected  error  in  the  yield  prediction?  Can  the  prediction  of  the 
yield  response  be  enhanced?  If  more  experiments  were  needed,  which  combinations  of 
the  two  design  variables  would  be  desirable?  Are  some  of  the  experiments  suspect 
(possibly  due  to  human  error)  and  should  be  removed  as  “outliers”?  Response  surface 
techniques  answer  these  questions  and  deal  with  these  issues. 
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Response  surface  methodology  initially  was  developed  and  described  in  a paper 
by  Box  and  Wilson  (1951),  and  further  investigated  in  subsequent  papers  Box  and  Hunter 
(1957)  and  Box  and  Draper  (1959).  It  can  be  defined  as  “ a group  of  statistical  techniques 
for  empirical  model  building  and  model  exploitation"  (Box  and  Draper,  1987),  or 
similarly  as  ‘'a  collection  of  statistical  and  mathematical  techniques  useful  for  developing, 
improving,  and  optimizing  processes”  (Myers  and  Montgomery,  1995). 

Papers  and  the  books  devoted  to  the  topic  of  RS,  Myers  and  Montgomery  (1995), 
Box  and  Draper  ( 1987)  and  Khuri  and  Cornell  (1996)  focus  mainly  on  physical 
experiments  and  how  to  model  the  results  from  field  tests  in  order  to  achieve  maximum 
information  from  the  data  collected.  Hill  and  Hunter  (1966)  present  several  examples  of 
early  applications  of  response  surface  methodology  in  a review  paper. 

Because  of  the  increasing  commonality  between  physical  and  numerical 
experiments,  these  techniques  or  methodologies  have  been  extended  successfully  to 
engineering  optimization  based  on  computer  simulations.  First,  like  physical  experiments 
numerical  simulations  such  as  finite  element  analyses,  can  be  expensive  in  terms  of  time 
and  cost,  especially  in  terms  of  model  preparation  and  validation.  Second,  in  performing 
a large  number  of  experiments  needed  for  optimization  there  is  usually  an  advantage  in 
performing  them  in  batches.  This  is  true  for  the  farmer  who  may  need  to  wait  for  several 
months  for  the  results  of  an  experiment.  It  is  true  for  the  engineer  in  a laboratory  because 
necessary  equipment  may  not  always  not  be  available,  and  therefore  experiments  need  to 
be  scheduled  in  batches  when  the  equipment  is  available.  Similarly,  multiple  numerical 
simulations  can  be  performed  at  off  peak  hours,  at  night  or  during  weekends  when 
computers  are  rarely  used.  In  addition,  the  availability  of  parallel  computers  naturally 
allows  analyses  to  be  performed  in  batches  on  different  nodes  of  a parallel  computer 
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rather  than  one  at  the  time  ( Balabanov  el  cil,  1996).  Third,  like  physical  experiments, 
complex  numerical  simulations  often  acquire  numerical  noise  from  different  sources. 
However  some  differences  remain  between  physical  and  computational  experiments, 
perhaps  the  most  important  one  is  that  the  repetition  of  a numerical  experiment  usually 
gives  the  same  result,  while  this  is  far  from  true  for  a physical  experiment. 

Response  surface  methodology  can  be  used  to  construct  global  and  mid-range 
approximations  to  functions  in  structural  optimization.  Traditionally,  linear  and  quadratic 
polynomials  are  used.  Because  structural  optimization  requires  expensive  function 
evaluations,  it  is  important  to  construct  accurate  function  approximations  so  that  rapid 
convergence  may  be  achieved.  Toropov  (1989)  investigated  the  role  of  the  choice  of 
approximating  functions  on  the  accuracy  of  the  approximations.  Different  techniques  to 
find  the  region  of  interest  containing  the  optimal  design,  and  techniques  for  finding  more 
accurate  approximations  also  were  presented  by  Roux  el  al.  ( 1 998).  Roux  and  co-workers 
solved  the  weight  minimization  problem  for  two-,  three-,  and  ten-member  trusses  using 
response  surface  methodology  and  compared  their  results  with  solutions  in  the  literature 
( Arora , 1989). 

The  authors  found  that  the  accuracy  of  the  response  surface  is  influenced  strongly 
by  the  number  and  the  location  of  data  points  used  to  construct  the  response  surface 
approximation.  However,  ultimately  the  accuracy  that  can  be  achieved  is  limited  by  the 
choice  of  the  approximating  functions  and  the  design  domain  size.  For  a successful  use  of 
response  surface  methodologies,  the  authors  advocate  as  the  most  important  features  the 
use  of  engineering  knowledge  in  choosing  the  approximating  functions  and  the  use  of  as 
many  constraints  as  possible  in  order  to  reduce  the  size  of  the  design  domain.  This  so- 
called  reasonable  design  space  approach  is  discussed  further  in  the  next  section. 
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Venter  et  al.  (1996)  demonstrated  the  use  of  dimensional  analysis  for  reducing  the 
dimension  of  the  design  domain  in  their  weight-optimization  of  a stepped  plate.  The 
authors  reduced  the  number  of  design  variables  in  their  response  surface  approximation 
by  using  variables  that  are  intrinsic  to  the  problem,  which  were  identified  by 
nondimensional  analysis  of  the  governing  equations.  In  contrast  with  Roux  et  al,  and 
Venter  et  al.  used  higher-order  polynomials  (cubic  and  quartic  instead  of  the  more 
traditional  quadratic)  to  achieve  greater  accuracy  for  the  stress  and  buckling  constraints. 

Several  other  examples  of  higher-order  polynomials  are  available  in  the  literature. 
Van  Houten  et  al.  (1995)  for  example  used  a fifth  order  polynomial  response  surface  to 
approximate  the  drag  coefficient  in  an  aerodynamic  shape  optimization  as  a function  of 
two  design  variables.  Unfortunately  response  surface  techniques  that  use  higher-order  or 
even  quadratic  polynomials  become  difficult  to  implement  as  the  number  of  design 
variables  increases  since  the  number  of  data  required  to  fit  the  response  surface  becomes 
prohibitively  large.  For  problems  with  four  or  more  design  variables  Kok  et  al.  (1998) 
advocate  the  use  a series  of  local  linear  response  surface  approximations  until 
convergence  to  an  optimum  is  reached.  However  this  approach  suffers  from  many  of  the 
disadvantages  of  the  local  approximations  methods.  In  order  to  be  able  to  use  response 
surface  techniques  for  high-dimensional  problems  another  use  of  response  surface  called 
'‘multifidelity  response  surface  methodology”  is  gaining  popularity. 


Multifidelity  Response  Surfaces 

Most  applications  of  response  surface  methodologies  use  a region  in  a design 
space  that  is  a box  defined  in  terms  of  lower  and  upper  bounds  of  the  design  variables. 
However,  unless  it  is  possible  to  analyze  designs  at  least  at  all  the  vertices  of  the  box,  the 
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process  of  approximation  inside  the  box  can  entail  extrapolation  rather  than  interpolation, 
with  an  attendant  of  loss  of  accuracy.  With  an  n-dimensional  box  having  2"  vertices,  it 
becomes  impractical  to  evaluate  the  designs  at  all  the  vertices  for  values  of  n of  the  order 
of  1 0 or  more.  Moreover,  response  surface  approximation  may  not  guarantee  sufficient 
accuracy  even  when  limited  to  interpolation  since  complex  response  patterns  are  often 
difficult  to  approximate  as  low  order  polynomials.  A different  way  of  using  response 
surfaces  is  gaining  popularity  as  a solution  to  this  problem.  Inexpensive  lower-fidelity 
models  often  are  available  to  the  designer,  and  strategies  can  be  devised  that  use  the 
lower  fidelity  methods  to  produce  accurate  approximations  at  reasonable  computational 
cost . The  combined  use  of  lower  and  higher-fidelity  methods  and  response  surfaces  is 
referred  to  as  “multifidelity”  methods. 

Lower-fidelity  models  can  be  obtained  either  by  simplifying  the  analysis  model 
(e.g.,  by  using  a coarser  FE  mesh  discretization)  or  by  using  a simpler  model  (e.g.,  finite 
strip  instead  of  FE).  Usually  the  lower-fidelity  models  vary  with  the  design  variables  in  a 
manner  similar  to  the  high-fidelity  model,  thus  providing  a good  basis  for  the  high  quality 
approximations.  Some  of  the  multifidelity  methods  use  the  high-fidelity  models  to 
“correct”  the  lower-fidelity  model  rather  than  to  approximate  directly  the  expensive 
higher-fidelity  model.  Toropov  and  Markine  ( 1 996)  classify  multifidelity  response 
surface  methods  as  multiplicative  or  additive.  For  multiplicative  cases,  correction 
response  surfaces  are  used  to  multiply  the  low-fidelity  result.  In  the  additive  cases  the 
correction  response  surface  is  added  to  the  results  obtained  from  the  low-fidelity  method. 

In  this  spirit,  Mason  et  al.  (1994)  used  quadratic  correction  response  surface 
techniques  to  design  channel  frames  subject  to  stress  constraints  where  the  lower-fidelity 
models  used  2-D  FE  models,  and  high-fidelity  models  used  3-D  FE  models.  A low- 
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fidelity  model  also  can  be  a closed-form  approximation.  Vital i et  al.  (1997)  used  closed- 
form  beam  and  plate  solutions  to  predict  stresses  and  buckling  loads  in  a hat-stiffened 
panel,  using  the  STAGS  ( Rankin  et  al.,  1998)  FE  code  for  correction.  Constant  response 
surface  were  constructed  for  the  ratios  of  the  stresses  or  buckling  loads  from  high-fidelity 
and  low-fidelity  models  at  pre-selected  design  points.  Stresses  and  buckling  loads  for 
other  points  were  approximated  by  multiplying  the  lower  fidelity  results  by  the  response 
surface. 

Instead  of  using  a constant  response  surface,  Venkataraman  and  Haftka  ( 1 998) 
used  a linear  correction  response  to  predict  the  buckling  loads  of  cylindrical  panels.  The 
authors  used  the  PANDA2  ( Bushnell , 1987)  program  as  their  low-fidelity  model  and  the 
STAGS  FE  program  as  their  high-fidelity  model.  PANDA2  is  a panel  design  program 
that  uses  closed  form  or  Rayleight-Ritz  type  solutions  to  obtain  failure  loads.  A quadratic 
response  surface  in  five  design  variables  was  created  by  using  many  PANDA2  analyses 
and  then  a linear  correction  response  surface  approximation  was  fit  to  the  ratios  of  the 
STAGS  analyses  and  the  PANDA2  analyses  at  few  selected  design  points.  Buckling 
loads  for  designs  in  the  design  domain  then  could  be  obtained  by  multiplying  the 
quadratic  response  surface  approximation  and  the  linear  correction  response  surface.  The 
buckling  load  approximations  were  used  as  constraints  for  weight  optimization  of 
cylindrical  shells. 

Balabanov  et  al  ( 1 998)  compared  the  results  obtained  using  both  the 
multiplicative  and  the  additive  correction  response  surface  approximations  in  predicting 
optimum  wing  bending  material  weight  of  a High  Speed  Civil  Transport  (HSCT)  as  a 
function  of  the  aircraft  geometry.  The  approximations  used  thousands  of  structural 
optimizations  of  computationally  inexpensive  coarse  finite  element  models  to  produce  a 
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quadratic  response  surface  model  in  the  29  design  variables,  and  then  about  a hundred 
structural  optimizations  of  refined  FE  models  were  used  to  construct  linear  correction 
response  surfaces  approximations.  They  found  that  the  linear  correction  response  surfaces 
reduced  the  error  of  the  quadratic  response  surface  approximations  based  on  coarse 
models  by  more  than  half.  Moreover,  most  of  the  correction  could  be  obtained  with  only 
a constant  correction  with  linear  terms  adding  only  marginal  improvements.  The  additive 
correction  response  surface  performed  better  than  the  multiplicative  one.  However,  the 
authors  suggested  that  the  advantage  of  the  additive  correction  response  surface  is  not  a 
general  rule  but  possibly  is  a feature  of  the  specific  problem  that  they  considered. 

A different  strategy  in  combining  low-fidelity  and  high-fidelity  models  was 
followed  by  Kaufman  et  al.  (1996)  and  Balabanov  et  al.  (1999)  when  searching  for  the 
optimal  takeoff  gross  weight  for  the  HCST.  This  strategy  is  called  “reasonable  design 
space  approach”.  The  reasonable  design  space  approach  seeks  inexpensive  constraints 
that  eliminate  from  consideration  large  portions  of  the  design  box  defined  by  the  upper 
and  lower  limits  on  the  design  variables.  Such  constraints  may  come  from  simple 
geometric  constraints  that  prevent  combinations  of  design  variables  resulting  in 
unreasonable  geometry  configurations  for  the  aircraft.  Low-fidelity  methods  were  used  to 
estimate  the  performance  of  candidate  designs  via  aerodynamic  constraints  and 
performance  constraints.  The  low-fidelity  methods  may  not  have  provided  accurate 
estimates  of  performance  but  they  could  identify  designs  with  such  inferior  performance 
that  they  could  be  discarded  even  based  on  very  approximate  results.  The  points  that 
corresponded  to  unreasonable  aircraft  configurations  were  not  simply  removed  but  they 
were  moved  toward  the  center  of  the  design  space  until  geometric,  performance  and 
aerodynamic  constraints  were  met  based  on  the  low-fidelity  method.  The  ratio  of  the 
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volume  of  the  initial  design  space  box  to  the  volume  of  the  final  reasonable  design  space 
for  the  25  design  variables  considered  was  about  10  . After  the  reasonable  design  space 
was  obtained  moving  the  unreasonable  design  points  tower  the  center  of  the  design  space 
using  the  lower  fidelity  analysis  about  1000  high-fidelity  analysis  were  performed  and  a 
response  surface  created  using  only  the  high-fidelity  data.  Using  this  last  response  surface 
the  authors  were  able  to  find  an  aircraft  configuration  that  was  18.3%  lighter  than  the 
baseline  configuration. 

Low-fidelity  methods  can  also  be  used  to  reduce  the  number  of  terms  in  a 
response  surface  and  then  the  reduced  response  surface  can  be  fitted  using  a smaller 
number  of  high-fidelity  analyses.  This  approach  was  followed  by  Knill  et  al.  (1999)  to 
efficiently  implement  aerodynamic  prediction  from  Euler  solutions  into  a 
multidisciplinary  design  optimization  study  of  the  HSCT.  Statistical  techniques  were 
applied  to  linear  theory  response  surface  models  to  reveal  the  terms  that  had  a significant 
impact  on  calculating  the  desired  aerodynamic  quantities.  Terms  that  had  little  or  no 
effect  could  be  removed  without  degrading  the  accuracy  of  the  response  surface  models. 
Only  important  terms  were  retained  when  the  reduced-terms  response  surface  models 
were  constructed  using  Euler  solutions  with  viscous  drag  correction,  thereby  requiring 
lewer  computer  fluid  dynamics  (CFD)  analyses  to  compute  the  coefficients  of  the 
models.  The  reduced-term  models  were  created  as  multiplicative  corrections  of  the  full- 
term  linear  theory  response  surface  models.  In  such  approach  there  was  the  possibility 
that  some  important  non-linear  effect  could  be  missed  but  this  did  not  seem  to  happen.  By 
using  the  low-fidelity  methods  to  discard  non  important  terms  the  authors  could  save  255 
CPU  hours  out  of  392  required  for  the  full-term  response  surface  while  maintaining  the 
same  level  of  accuracy. 
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Multiple  Models  in  Optimization 

The  increasing  complexity  of  engineering  systems  has  sparked  increasing  interest 
in  multidisciplinary  optimization  (MDO).  MDO  can  be  described  as  a methodology  for 
the  design  of  systems  where  the  interaction  between  several  disciplines  must  be 
considered,  and  where  the  designer  is  free  to  significantly  affect  the  system  performance 
in  more  than  one  discipline.  Two  are  the  main  challenges  of  MDO:  computational 
expense  and  organizational  complexity.  These  challenges  are  summarized  in  the 
following  based  on  Sobieszczanski-Sobieski  and  Haftka,  (1997)  paper. 

The  interdisciplinary  coupling  inherent  in  MDO  tends  to  present  additional 
challenges  beyond  those  encountered  in  a single-discipline  optimization.  It  increases 
computational  burden,  and  it  also  increases  complexity  and  creates  organizational 
challenges  for  implementing  the  necessary  coupling  in  software  systems.  Since  solution 
times  for  most  analysis  and  optimization  algorithms  increase  at  a superlinear  rate,  the 
computational  cost  of  MDO  is  usually  much  higher  than  the  sum  of  the  costs  the  single- 
discipline optimizations  for  the  disciplines  represented  in  the  MDO.  Additionally,  even  if 
each  discipline  uses  linear  analysis  methods,  the  combined  system  might  require  costly 
non-linear  analysis.  For  example,  linear  aerodynamics  may  be  used  to  predict  pressure 
distributions  on  a wing,  and  linear  structural  analysis  may  be  then  used  to  predict 
displacements.  However  the  dependence  of  the  pressure  on  the  displacements  may  not  be 
linear.  Finally,  for  each  disciplinary  optimization  we  may  be  able  to  use  a single- 
objective function,  but  for  the  MDO  problem  we  may  need  to  have  multiple  objectives 
with  an  attendant  increase  in  optimization  cost. 

MDO  also  faces  formidable  organizational  challenges.  The  analysis  codes  for 
each  discipline  have  to  be  made  to  interact  with  one  an  other  for  the  purpose  of  system 
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analysis  and  system  optimization.  Data  transfers  among  modules  of  an  MDO  problem 
correspond  to  the  internal  couplings  of  the  system.  These  data  transfers  may  require  data 
processing  that  may  become  a costly  overhead.  For  example,  if  the  system  is  a flexible 
wing,  the  aerodynamic  pressure  reduced  to  concentrated  forces  at  the  aerodynamic  model 
grid  points  on  the  wing  surface  has  to  be  converted  to  the  corresponding  concentrated 
loads  acting  on  the  structure  finite-element  nodal  points.  Conversely,  the  finite  element 
nodal  structural  displacements  have  to  be  entered  in  the  aerodynamic  model  grid  as  shape 
corrections.  The  volume  of  data  transferred  in  such  couplings  affects  efficiency  directly 
in  terms  of  I/O  cost.  Additionally  many  solution  procedures  require  the  derivatives  of 
these  data  with  respect  to  design  variables,  so  that  a large  volume  of  data  also  increases 
computational  cost.  To  decrease  these  costs  response  surface  approximations  can  be  used. 

In  the  above  wing  example  one  may  represent  the  pressure  distribution  and  the 
displacement  fields  by  response  surface  approximations,  where  the  approximating 
functions  constituting  the  response  surface  are  chosen  by  the  designer  beforehand  based 
on  engineering  knowledge.  In  this  way  only  the  coefficients  of  the  approximating 
functions  have  to  be  transferred  instead  of  the  large  volumes  of  the  discrete  load  and 
displacement  data.  An  example  of  such  a condensation  for  supersonic  transport  design 
was  reported  in  Unger  et  al.  (1992). 

Direct  coupling  of  the  optimizer  to  a multidisciplinary  analysis  may  be 
impractical  for  various  reasons.  First,  for  moderate  to  large  number  of  design  variables, 
the  number  of  evaluations  of  objective  function  and  constraints  required  by  the  optimizer 
is  high.  Often  we  cannot  afford  to  execute  such  a large  number  of  exact  MDO  analyses  in 
order  to  provide  the  evaluation  of  the  objective  function  and  constraints.  Second,  often 
the  different  disciplinary  analyses  are  executed  on  different  machines,  possibly  at 
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different  sites,  and  communication  with  a central  optimizer  can  become  impossible. 

Third,  some  disciplines  may  produce  jagged  or  noisy  responses  as  a function  of  the 
design  variables  ( e.g Giuntci  et  al.  (1994).  For  all  of  these  reasons  often  the  optimizer  is 
connected  to  easy-to-calculate  approximations  of  the  objective  function  and  constraints. 
Response  surface  approximations  are  particularly  suitable  for  this  task.  For  MDO 
problems  response  surface  techniques  also  provide  a convenient  representation  of  the 
data  form  one  discipline  to  other  disciplines  and  to  the  system.  Sellar  et  al.  (1996) 
designed  an  autonomous  hovercraft  where  four  disciplines  used  four  different  codes  that 
were  made  to  communicate  through  response  surface  approximations. 

Response  surface  approximations  are  not  only  used  as  a mean  for  communication 
between  different  disciplinary  codes,  but  can  serve  also  as  a communication  tool  between 
codes  of  the  same  discipline  representing  multiple  models  in  that  discipline.  For  example, 
simple  but  flexible  interface  between  the  global  and  the  local  design  optimization  codes 
is  often  needed  when  designing  a complex  structure,  and  response  surface  can  serve  for 
this  purpose.  Ragon  et  al.  (1997)  approximated  the  optimal  panel  weight  obtained  from 
the  local  design  code  with  an  response  surface  for  designing  wing  structure.  This 
approximation  was  then  used  as  a constraint  in  the  global  wing  approximation.  Liu  and 
Haftka  ( 1 998)  used  a similar  approach  for  transmitting  optimal  buckling  loads  from  the 
panel  level  to  the  wing  level.  Venkataraman  and  Haftka  (1997)  approached  a similar 
problem  from  the  opposite  end  when  designing  the  structure  of  a laminated  composite 
liquid  hydrogen  tank.  Approximations  in  the  form  of  response  surface  were  constructed 
for  the  global  buckling  and  the  bowing  displacement  from  the  global  analysis  and  then 
the  response  surface  approximations  were  used  as  constraints  in  local  optimization. 
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Response  surface  approximations  have  recently  gained  popularity  as  a simple  way 
to  facilitate  communications  between  specialists  on  the  design  team.  In  this  sense,  these 
techniques  are  becoming  one  of  the  means  to  meet  the  organizational  challenge  of  MDO 
(Tai  et  al.,  1995  and  Giunta  et  al.,  1995).  The  use  of  response  surface  approximations  in 
general  often  allows  dealing  in  a better  way  with  organizational  boundaries.  The 
approximations  used  for  each  discipline  can  be  generated  by  specialists  of  the  discipline. 
The  specialist  can  tailor  the  approximation  to  special  features  of  that  discipline  and  to  the 
particulars  of  the  application.  When  response  surface  are  used,  the  creation  of  the  various 
disciplinary'  approximations  can  be  performed  ahead  of  time,  minimizing  the  interaction 
of  the  optimization  procedure  with  the  various  disciplinary  software. 

Visualization  and  Response  Surfaces 

Global  approximation  methods  in  general  and  response  surface  techniques  in 
particular  offer  the  designer  a global  picture  of  the  problem,  pointing  out  tradeoffs  and 
particularly  promising  regions  within  the  design  space.  Visualization  techniques  help  the 
designer  take  advantage  this  information.  There  is  a general  agreement  that  visualization 
has  an  important  role  in  computational  science  and  engineering,  allowing  scientists  and 
designers  to  gain  an  understanding  that  is  otherwise  not  possible.  Coupling  visualization 
techniques  and  response  surface  can  help  in  acquiring  a deeper  understanding  of  the 
problem  at  hand.  Knill  et  al.  (1999)  used  response  surface  techniques  to  optimize  the 
takeoff  gross  weight  (TOGW)  of  an  HSCT  aircraft.  Different  aircraft  configurations  were 
obtained  using  1 0 design  variables.  The  authors  found  that  depending  on  the  starting 
points  their  optimization  converged  to  two  slightly  different  HSCT  configurations  that 
were  2000  lb.  apart.  The  question  of  why  the  optimizer  was  not  able  to  always  reach  the 
lowest  TOGW  configuration  was  sought  by  visualizing  the  design  space.  Figure  1 shows 
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a section  of  a two-dimensional  slice  through  the  multidimensional  design  space  using  the 
first  and  the  second  optimum  found  by  Knill  and  co-workers,  and  a sub-optimal  feasible 
point. 
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Figure  1 : Visualization  of  the  design  domain  as  in  Knill  et  al. 


The  remaining  points  in  the  plane  were  created  by  linear  combinations  of  the 
three  points.  In  Figure  1 the  circles  represent  design  points.  Open  circles  represent 
feasible  points  while  filled  circles  represent  points  at  which  the  constraints  are  violated. 
The  value  of  the  objective  function  obtained  from  a response  surface  in  the  10  design 
variables  is  indicated  by  the  shaded  contour  plots.  The  curved  lines  represent  the 
boundaries  of  four  constraints.  Prior  to  creating  the  visualization,  it  was  not  recognized 
by  Knill  et  al.  that  the  constraints  break  the  design  space  into  disjoint  (at  least  in  some 
hyper-planes)  regions  of  feasible  points.  Only  after  the  visualization  of  the  design  domain 
and  objective  function  in  a sub-domain  around  the  two  optima  was  obtained,  it  was 
possible  to  understand  why  the  optimum  depended  on  the  starting  point.  Visualization  in 
conjunction  with  response  surface  and  optimization  provide  understanding  of  the 
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optimization  process  and  the  tradeoffs  involved.  They  also  have  the  potential  to  provide 
guidance  by  an  experienced  engineer  when  the  optimizer  experiences  difficulties. 

The  HSCT  aircraft  is  also  the  test  bed  for  a specific  visualization  software, 
VIZCRAFT  (Gael  et  ai,  1999)  that  is  being  developed  at  Virginia  Tech  at  the  MAD 
Center.  VIZCRAFT  is  based  on  two  basic  tools  to  better  manage  the  information 
available.  The  first  tool  permits  the  user  to  quickly  evaluate  the  quality  of  a design  with 
respect  to  its  objective  function  and  constraints.  Each  time  the  value  of  a design  variable 
is  modified,  the  HSCT  planform  is  immediately  updated  and  displayed  to  reflect  the  new 
geometry,  and  so  are  the  value  of  the  objective  function  and  of  the  constraints.  However, 
in  order  to  have  a quick  prediction  of  the  objective  function  and  of  the  constraints,  the 
VIZCRAFT  program  does  not  perform  any  additional  analysis  but  relies  on  response 
surface  based  on  previously  calculated  data.  A costly  evaluation  of  the  objective  function 
and  constraints  by  CFD  models  of  the  aircraft  is  performed  only  if  the  user  explicitly 
requests  it.  When  the  new  evaluation  requested  by  the  user  is  performed  then  the  results 
are  added  to  the  database  on  which  the  response  surface  for  quick  calculations  are  based 
on.  Parallel  coordinates  is  the  second  tool  used  by  VIZCRAFT  to  visualize  the  results  of 
several  HSCT  configurations  in  a high  dimensional  space.  This  technique  allows  the  user 
to  visualize  several  different  HSCT  design  variable,  constraints  and  objective  function 
vectors  at  the  same  time  and  enables  the  user  to  recognize  potential  promising  designs. 

Malone  and  Woyak  (1999)  presented  a visualization  software,  called 
MODELCENTER  (Malone  and  Woyak,  1998),  that  help  aircraft  designers  to  combine 
analyses  from  several  design  disciplines  into  one  model,  and  visually  observe  their 
interaction.  As  the  optimization  proceeds  the  designer  can  halt  the  optimization  and 
display  the  aircraft  geometry  on  the  screen  with  a simple  click.  Moreover  charts  in  two 
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and  three  dimension  of  any  parameter  combinations  can  be  displayed  so  that  the  designer 


can  follow  the  optimization  as  it  progresses  and  perhaps  visually  detect  if  some 
anomalies  occur.  In  Figure  2 a snap  shot  of  the  visualization  as  it  is  furnished  by 
MODELCENTER  is  presented. 
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Figure  2:  MODELCENTER  visualization  of  an  aircraft  optimization 


The  potential  of  the  software  developed  was  shown  in  the  cost  study  of  a tighter. 
Very  simple  aerodynamic  analysis  tool  were  used  to  estimate  the  lift  and  drag 
performance  of  the  wings  and  a response  surface  was  used  to  estimate  the  development 
and  manufacturing  cost  of  the  aircraft.  The  software  used  the  knowledge  about  the 
components  in  a “smart-’  way,  updating  only  the  information  about  the  components  that 
were  likely  to  have  a grater  impact  on  the  cost  of  the  aircraft. 

Parallel  Computations 

Modern  aerospace  vehicle  design  requires  the  interaction  of  multiple  disciplines, 
traditionally  processed  in  a sequential  order.  MDO  is  evolving  towards  methods  capable 
of  replacing  the  traditional  sequential  methodology  by  concurrent  algorithms,  with  both 
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an  overall  gain  in  product  performance  and  a decrease  in  design  time.  Two  methods  are 
available  to  the  designer  when  trying  to  exploit  parallelism  in  optimization,  coarse-grain 
and  fine-grain  parallelism.  Coarse-grain  parallelism  involves  the  concurrent  execution  of 
multiple  independent  function  evaluations  while  fine-grain  parallelism  involves 
computing  the  basic  computational  steps  of  an  algorithm  (i.e.  the  internal  algebra)  in 
parallel.  In  general  coarse-grain  parallelism  requires  very  little  inter-processor 
communication  while  the  fine-grain  parallelism  involves  much  more  data  transfer. 

An  MDO  paradigm  based  on  coarse-grain  parallel  computing,  variable- 
complexity  modeling  and  multipoint  response  surface  approximations  was  presented  by 
Burgee  et  al.  (1996).  This  paradigm  interleaves  the  disciplines  at  one  level  of  complexity 
and  processes  them  hierarchically  at  another  level  of  complexity,  achieving  parallelism 
within  the  disciplines  rather  then  across  disciplines.  The  coarse-grained  parallelism  was 
chosen  over  the  fine-grained  parallelism  because  it  allows  orchestrating  the  process  with 
minimal  knowledge  of  the  individual  disciplines  involved:  aerodynamics  and  structures. 
The  overall  parallelization  task  had  two  distinctive  sub-tasks:  parallelizing  the  individual 
steps  of  the  MDO  problem  and  linking  everything  together  so  that  the  entire  MDO 
becomes  a single  parallel  problem.  The  last  sub-task  is  particularly  time  consuming 
because  an  MDO  problem  typically  involves  programs  in  several  languages,  extensive 
I/O  for  some  program  components,  manual  transfer  of  data  between  formats  and  between 
machines,  and  human  intervention  to  interpret  the  output  and  tune  the  input  parameters. 

Burgee  and  co-workers  used  parallelization  extensively  while  trying  to  optimize 
the  TOGW  of  an  HSCT  aircraft.  First  a master-slave  paradigm  was  implemented  such 
that  each  node  of  the  parallel  computer  performed  its  own  optimization  to  find  a D- 
Optimal  set  of  points  where  to  construct  aerodynamic  and  structural  RS.  Then  the  coarse- 
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grained  parallelization  of  the  aerodynamic  analysis  modules  made  use  of  a master-slave 
paradigm  on  the  Intel  Paragon,  whereby  one  designated  master  node  controlled  the  data 
transfer  and  the  I/O  of  the  remaining  nodes.  Once  all  the  analyses  were  obtained  a 
response  surface  approximation  for  the  aerodynamic  response  was  constructed  as  a 
function  of  the  design  variables. 

The  parallelization  of  the  structural  analyses  performed  by  Burgee  et  al.,  followed 
a similar  master-slave  paradigm,  where  the  structural  code  used  was  GENESIS  (VMA 
Engineering.  1 997).  Structural  analyses  were  performed  in  parallel  on  the  Intel-Paragon 
at  some  D-Optimal  points  previously  chosen  and  then  a response  surface  approximation 
for  the  structural  response  was  constructed.  However,  at  first  the  speed-up  achieved  could 
not  be  increased  over  2.3  no  matter  how  many  nodes  were  used.  The  speed-up  is  defined 
as  the  ratio  of  the  computation  time  obtained  with  a parallel  computer  to  the  time 
required  to  solve  the  problem  the  same  problem  with  only  one  processor.  In  theory  the 
maximum  speed-up  achievable  should  be  equal  to  the  number  of  nodes  used  in  the 
parallel  computation.  The  reason  of  this  poor  result  even  when  20  nodes  were  used,  was 
due  to  the  huge  amount  of  I/O  that  GENESIS  required.  For  this  reason,  GENESIS  was 
modified  in  order  to  diminish  the  volume  of  I/O  operation  and  ultimately  a speed-up 
factor  of  1 1 .7  when  using  20  nodes  was  achieved.  This  was  a substantial  improvement 
but  it  was  still  not  totally  satisfactory.  In  order  to  achieve  greater  speed-ups  the  authors 
recommend  further  modification  to  the  GENESIS  program  in  order  to  reduce  even  more 
of  I/O  operations  needed.  Similar  results  were  reported  by  Kaufman  et  al.  (1996)  in  their 
study  of  the  TOGW  of  an  HSCT  aircraft  using  parallel  computers. 

Krasteva  et  al.  (1999)  continued  the  studies  of  Burgee  et  al.  using  different 
schemes  for  the  parallelization  process  in  order  to  achieve  better  speedups.  In  particular  it 
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was  noted  that  the  master-slave  scheme  used  by  Burgee  et  al.  produced  a severe  load 
imbalance  among  the  nodes  where  the  total  idle  time  amounted  to  one  half  of  the  total 
processing  time.  In  particular  Krasteva  et  al.  implemented  two  dynamic  balancing 
strategies  so  that  the  load  could  be  effectively  redistributed  among  processors.  Krasteva 
and  co-workers  implemented  two  dynamic  loading  algorithms  to  achieve  better  speedups 
on  and  Intel  Paragon:  random  polling  (RP)  and  global  round  robin  with  message 
combining  (GRR-MC).  Unlike  the  strategies  used  by  Burgee  et  al.  the  two  dynamic 
loading  strategies  considered  by  Krasteva  and  co-workers  conserved  linearity  in  speedups 
as  more  processors  were  added.  In  other  words  using  the  RP  and  the  GGR-MC  strategies 
Krasteva  and  co-workers  were  able  to  halve  the  processing  time  when  the  number  of 
nodes  was  doubled. 

Eldred  and  Schimel  ( 1 999)  presented  a study  where  they  advocated  the  use  of  a 
mix  of  coarse-grained  and  fine-grained  parallelism  for  optimal  parallelization.  The 
authors  recommend  that  when  using  parallel  computers  coarse-grained  parallelism  should 
be  used  as  much  as  possible  since  minimal  inter-processor  communication  exists  between 
processors.  However,  when  using  massively  parallel  computers  there  may  be  fewer 
analyses  to  perform  than  nodes,  or  some  nodes  may  be  idle  for  other  reasons.  In  this  case 
it  is  suggested  that  a mix  of  fine-grained  and  coarse-grained  should  be  used. 

Coarse-grain  parallelism  was  also  exploited  by  Kroo  et  al.  (1994)  in  their  MDO 
study  of  an  aircraft  using  the  collaborative  optimization  method.  Collaborative 
optimization  involves  the  decomposition  of  the  design  problem  in  sub-problems  that  can 
then  be  run  in  parallel  on  a parallel  machine.  Using  this  strategy  a system-level  optimizer 
is  used  to  pass  parameters  to  the  different  sub-problems.  These  parameters  are  adjusted 
by  the  system  level  optimizer  to  maximize  the  objective  function  while  satisfying  the 
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system-level  constraints.  Using  coarse-grain  parallelization  and  the  collaborative 
optimization  strategy  Kroo  and  co-workers  were  able  to  decrease  the  computational  time 
by  69%  for  the  problem  they  considered. 

Numerical  Noise 

The  solution  of  optimization  problems  is  more  difficult  in  the  presence  of 
numerical  noise.  Noisy  behavior  of  the  objective  function  and  of  constraints  can  occur  as 
a result  of  incomplete  convergence  of  iterative  processes,  the  use  of  adaptive  numerical 
algorithms,  round  off  errors,  discrete  representation  of  continuous  physical  objects,  and 
discontinuities  in  the  functions  that  are  being  calculated.  Such  noise  is  typically 
manifested  as  a high-frequency,  low  amplitude  variation  in  the  results  obtained  from 
computer  simulations  as  the  design  parameters  vary.  When  numerical  optimization  is 
attempted,  this  oscillatory  behavior  can  create  numerous,  spurious  local  optima  and  can 
cause  slow  convergence  or  even  convergence  failure.  Giunta  et  al.  (1994)  faced  these 
issues  when  trying  to  optimizing  the  TOGW  of  an  HSCT  aircraft  due  to  noise  in 
aerodynamic  response.  Giunta  et  al.  successfully  applied  response  surface  to  remove 
noise  form  the  aerodynamic  response.  Figure  3 shows  an  example  of  the  noise  in  the 
volumetric  drag  coefficient  Cowave  as  function  of  the  wing. 
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Figure  3:  Cowave  noisy  behavior  from  Giunta  et  al. 

The  response  surface  approximation  used  and  shown  in  Figure  3 filters  out  the 
numerical  noise  present  in  the  data  while  preserving  the  overall  trend  in  the  variation  in 
C-DWave-  Although  the  Cowave  calculations  are  noisy,  the  numerical  noise  has  small 
magnitude  and  therefore  does  not  affect  much  the  Cowave  function  values.  However  it 
does  affect  greatly  the  calculation  of  derivatives.  From  Figure  3 it  is  clear  that  not  only 
the  derivatives  of  Cpwave  with  respect  to  the  wing  tip  spanwise  distance  are  inaccurate  but 
that  the  fluctuation  of  the  derivatives  also  creates  spurious  local  optima.  As  a 
consequence  traditional  derivative-based  optimization  algorithms  become  unreliable. 

When  a response  surface  approximation  to  the  Cowave  is  fitted  to  the  data  a 
smooth  response  is  obtained.  Since  physically  Cowave  should  change  smoothly  with  the 
span,  the  response  surface  is  likely  to  be  more  accurate  then  the  actual  aerodynamic  data. 
A similar  situation  where  the  response  surface  was  probably  more  accurate  than  the  data 
used  for  its  construction  was  encountered  by  Venter  et  al.  (1996)  as  shown  in  Figure  4b. 
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a)  b) 

Figure  4:Variable  thickness  plate  geometry  a)  and  noise  is  FE  analysis  b)  as 

in  Venter  et  al. 

Venter  et  al.  encountered  some  inconsistencies  in  the  data  obtained  from  the  finite 
element  model  (represented  by  the  dots  in  Figure  4b)  in  studying  a plate  with  variable 
thickness.  In  particular,  it  yielded  less  accurate  finite  element  models  for  plates  with 
values  of  a close  to  1 , where  a was  defined  as  the  ratio  of  the  thickness  of  the  thicker 
part  of  the  stepped  plate  to  the  thinner  one  (see  Figure  4a).  Note  that  for  a = 1 .0  the  non- 
dimensional  stress  should  be  equal  to  one.  The  inconsistencies  were  attributed  to  the 
mesh  generation  procedure  that  did  not  provide  finite  element  models  with  a consistent 
level  of  fidelity.  The  results  presented  in  Figure  4b  indicate  that  the  response  surface 
approximation  eliminated  these  inconsistencies,  and  indicate  that  not  all  differences 
between  predicted  and  numerical  response  values  should  be  considered  to  be  errors. 

Free  et  al.  (1987)  compared  derivative  based  optimization  algorithms  and 
response  surface  based  algorithm.  They  found  that  the  methods  used  were  comparable  in 
accuracy  and  computational  cost  when  no  noise  was  present,  but  that  response  surface 
based  optimization  consistently  performed  better  when  noise  was  present. 
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The  ability  of  response  surface  to  filter  out  with  noise  was  exploited  by  Etman  et 
al.  (1996)  in  a crash  worthiness  optimization  problem.  Physical  crash  tests  of  newly 
designed  vehicles  are  used  to  determine  the  vehicle’s  crash  worthiness.  However,  these 
physical  tests  are  very  costly,  and  therefore  car  producers  generally  tend  to  use  instead 
simulation  software,  such  as  MADYMO  (TNG  Road-Vehicles  Research  Institute , 1994) 
in  the  design  stage.  Although  crash  simulations  are  computationally  very  expensive,  they 
are  much  cheaper  than  physical  tests.  As  a result,  the  number  of  designs  that  can  be 
analyzed  is  limited.  Designers  are  urged  to  find  methods  that  allow  optimization  using  as 
few  crash  analyses  as  possible.  Etman  et  al.  found  that  the  behavior  of  injury  parameters 
as  a function  of  the  design  variables  was  not  smooth.  Figure  5 shows  the  head  injury 
criterion  (HIC)  as  a function  of  the  airbag  hole  size  when  all  the  other  parameters  are  held 
constant. 


Airbag  hole  diameter  [m] 

Figure  5:  Noisy  response  as  obtained  by  Etman  et  al.  for  the  head  injury  criterion  (HIC) 

as  a function  of  the  airbag  hole  diameter 

From  Figure  5,  it  is  clear  that  the  noise  caused  gradients  to  be  inaccurate,  thus 
rendering  traditional  derivative  based  optimization  algorithms  inappropriate  for  this 
problem.  Etman  and  co-workers  used  an  iterative  process  where  sequences  of  linear 
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response  surface  were  obtained  in  sub-domains  until  convergence  to  an  optimum  was 
achieved.  Six  design  variables  were  used  to  optimize  the  HIC  subject  to  injury  constraints 
on  head,  chest,  and  shoulders  produced  by  the  airbag  and  the  seat  belt.  Schoofs  et  al. 
(1992)  approached  a similar  problem  of  crash  worthiness  when  optimizing  the 
performance  of  a child  seat  as  a function  of  several  neck  injury  parameters.  In  contrast  to 
Etmcm  and  co-workers,  Schoofs  et  al.  used  quadratic  response  surface  to  approximate  the 
behavior  of  child  seat  through  the  entire  design  domain.  However  Schoofs  and  co- 
workers used  only  two  design  variables. 

Noisy  response  functions  were  also  encountered  by  Toropov  et  al.  (1996)  in  their 
study  of  a conical  continuous  fiber  reinforced  thermoplastic  (CFRTP)  material  shell 
under  compressive  loads.  Figure  6a  shows  the  finite  element  model  used  by  Toropov  and 
co-workers,  while  Figure  6b  shows  the  objective  function  for  the  conical  shell  as  a 
function  of  height  of  the  shell  (x/)  and  the  base  radius  (x?). 


a) 


b) 


Figure  6:  Finite  element  model  a)  and  noisy  response  b)  obtained  by  Toropov  at  al. 


Once  again  it  is  evident  from  Figure  6b  that  gradient  based  optimization 
algorithms  would  fail  to  reach  the  optimum.  The  use  of  response  surface  approximations 
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can  smooth  out  the  objective  function  providing  a more  accurate  prediction  than  the  FE 
analyses  themselves. 

Kaufman  et  al.  (1996)  encountered  significant  numerical  noise  in  the  optimization 
of  the  HSC'T  wing  structural  weight  material.  Variations  in  the  objective  function  due  to 
numerical  noise  amounted  up  to  as  much  as  27%.  It  is  sometimes  difficult  to  separate 
noise  generated  by  the  computer  simulations  from  fitting  errors.  Apparent  errors  in  the 
response  surface  approximations  must  be  carefully  interpreted.  Indeed  a substantial  part 
of  the  error  may  represent  a desirable  smoothing  of  response  surface  from  the  results  of 
the  structural  optimization.  In  other  words  when  dealing  with  noisy  functions,  response 
surface  can  be  more  accurate  than  the  computer  simulation  used  to  produce  the  data  on 
which  the  response  surface  approximation  is  based  on. 

Outliers 

In  general,  the  accuracy  of  response  surface  approximations  obtained  from  fitting 
experimental  data  (both  numerical  and  physical)  obtainable  can  be  improved  if  “outliers” 
can  be  detected  and  either  excluded  or  repaired.  Outliers  are  design  points  whose 
computed  or  measured  response  is  far  from  the  predicted  response  given  by  the  response 
surface  approximation,  and  their  number  is  usually  a small  fraction  of  the  number  of  data 
points  used  to  construct  the  response  surface  approximation.  Numerical  noise  is  often 
present  in  structural  optimization  problems  as  shown  in  a previous  section.  Flowever  it  is 
often  difficult  to  decide  if  outliers  are  due  to  numerical  noise,  modeling  errors  or  simply 
reflect  mistakes  in  the  program  or  data  preparation.  In  general,  simply  omitting  outliers 
from  the  regression  procedure  is  not  a good  practice.  Robust  regression  numerical 
procedures,  such  as  iteratively  reweighted  least  square  (IRLS),  can  be  used  to  identify 
outliers  in  the  data  set  and  then  once  identified  the  outliers  can  be  more  carefully  studied, 
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and  the  source  of  the  large  deviation  from  the  response  surface  approximation  found.  In  a 
laboratory  the  experiments  can  be  repeated  or  in  a computer  simulation  convergence 
parameters  can  be  tightened.  If  the  outliers  were  due  to  a mistake  or  convergence  failure 
they  can  be  repaired.  On  the  other  hand  if  the  response  values  remain  unaltered  even  after 
repair,  there  is  an  indication  that  bias  error  might  be  present  and  that  perhaps  a higher 
order  polynomial  or  different  shape  functions  should  be  considered  for  the  RS. 

Papila  and  Haftka  (1999)  repaired  outliers  when  fitting  response  surface 
approximations  to  optimize  the  wing  bending  material  weight  of  an  HSCT  aircraft.  They 
fitted  a quadratic  response  surface  approximation  in  five  design  variables  (2 1 terms)  to  43 
data  points  of  the  wing  bending  material  weight.  An  IRLS  procedure  identified  four  out 
of  the  43  HSCT  configurations  as  outliers.  Once  the  outliers  were  identified,  the  cause  of 
poor  accuracy  was  detected  as  optimization  premature  convergence.  The  outlier  points 
where  repaired  by  repeating  the  optimization  with  different  convergence  criteria,  leading 
to  more  accurate  approximation.  By  using  response  surface  and  IRLS  to  detect  outliers 
and  repair  them  the  authors  were  able  to  cut  by  half  the  predicted  errors  of  the  response 
surface  approximation. 

Response  surface  based  on  gradient  information 

Response  surface  techniques  have  been  originally  developed  for  fitting 

experimentally-hence  noisy  data.  Consequently,  these  methods  historically  focused  on 
fitting  function  data  without  any  attempt  to  use  derivative  data,  since  derivative  data  is 
rarely  available  from  physical  experiments.  In  computer  simulations,  derivatives  are  often 
available,  frequently  at  a small  computational  cost.  As  a consequence,  there  has  been  a 
growing  interest  in  considering  response  surface  that  incorporate  both  function  and 


derivative  data. 
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There  are  two  cases  where  using  derivative  data  is  particularly  attractive.  The  first 
case  is  when  the  response  surface  data  is  generated  in  the  course  of  an  optimization 
process.  Most  optimization  procedures  require  derivatives,  and  so,  as  the  optimization 
progresses,  function  and  derivative  data  is  accumulated  at  points  previously  visited  by  the 
optimization  procedure.  The  second  case  where  derivatives  are  attractive  is  when  the 
computational  cost  of  a derivative  is  much  lower  than  the  cost  of  a function  evaluation. 
This  happens,  for  example,  when  adjoint  methods  are  used  to  calculate  the  derivatives  of 
one  function  with  respect  to  many  design  variables.  Toomer  et  al.  (1998)  have  used  such 
adjoint  generated  gradients  of  aerodynamic  drag  for  response  surface  construction. 
However,  rather  than  using  derivatives  directly,  they  used  the  derivatives  to  generate 
extrapolated  function  values  at  nearby  points  and  then  used  these  additional  function 
values  with  standard  response  surface  methods. 

While  derivative  information  is  attractive  to  use  when  available  inexpensively,  it 
is  of  questionable  value  when  it  is  much  noisier  than  the  function  information  or  when 
derivatives  do  not  properly  exist  at  many  points  in  the  design  space.  Hence,  it  is  observed 
that  practical  use  of  derivative  data  for  response  surface  construction  is  potentially 
hindered  by  three  complications.  First  derivatives,  like  functions,  can  be  noisy  and 
derivatives  may  be  less  accurate  than  the  corresponding  functions.  Second,  although  the 
function  values  might  be  intrinsically  continuous,  the  derivatives  may  contain 
discontinuities.  Finally,  some  derivatives  may  be  calculated  inexpensively,  while  others 
may  be  equally  expensive  as  function  evaluations.  In  that  case,  adding  additional  function 
evaluations  probably  make  more  sense  than  adding  data  obtained  by  an  expensive 


sensitivity  analysis. 
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In  the  past,  several  papers  addressed  response  surface  approximation  constructed 
on  the  basis  of  both  function  values  and  derivatives.  There  were  mainly  two  motivations 
to  incorporate  information  on  derivatives.  First,  it  has  been  attempted  to  establish 
response  surfaces  which  provide  better  approximations  for  the  derivatives.  Second, 
reducing  computing  time  and  formulation  of  efficient  numerical  schemes  has  been  a the 
other  motivation,  especially  when  derivatives  can  be  calculated  cheaply. 

Sharifzadeh  et  al.  (1989)  considered  a manufacturing  process.  The  latter  is 
modeled  as  y = f(x)+e,  herej  denotes  a random  output  vector.  The  random  vector  z has 
zero  mean  and  is  mainly  determined  by  manufacturing  practice.  The  input  variables  x are 
given  as  x = xo+  tj,  with  xo  a designer  specified  vector  of  input  variables  and  77  are 
perturbations  independent  of  s and  with  zero  mean.  Sharifzadeh  and  co-workers  put  the 
main  focus  on  the  transmitted  variance,  var(y)  i.e.  variances  in  the  output  variables 
caused  by  perturbations  in  the  input  variables.  The  transmitted  variance  can  be 
approximated  by  VfT  var^)  Vf  It  is  clearly  seen  that,  this  calls  for  accurate  derivatives  of 

the  functions,  which  have  been  approximated  using  RS.  Sharifzadeh  et  al.  report  that 
application  of  a standard  response  surface  techniques  - based  on  function  values  only  - 
led  to  proper  results  for  function  values,  but  inaccurate  derivatives.  In  order  to  improve 
derivative  accuracy,  at  each  plan  point  a second  point  is  added  whose  coordinates  are 
slightly  perturbed  from  the  original  ones.  The  perturbation  in  taken  as  the  vector  ol 
standard  deviations  in  the  input  variables.  In  summary,  the  method  described  by 
Sharifzadeh  et  al.  attempts  to  improve  the  accuracy  of  derivatives  by  using  additional 
information  on  directional  derivative  data,  the  latter  is  incorporated  by  additional  function 
evaluations  in  the  neighborhood  of  regular  points.  The  authors  showed  that  the 
methodology  applied  was  successful  in  predicting  function  values,  but  success  for  the 
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transmitted  variability  was  more  limited.  It  seemed  to  be  inherently  much  harder  to  model 
variability  because  it  depends  on  derivatives  which  tend  to  be  less  like  polynomials  than 
the  functions  themselves. 

Response  surface  approximations  that  are  based  only  on  gradients  have  also  been 
used.  Ho  et  al.  (1991)  proposed  a so-called  gradient  surface  methodology  and  used  it  for 
unconstrained  optimization.  Within  the  optimization  process,  linear  approximations  are 
built  for  the  gradients  of  a response  function,  i.e.  approximations  to  the  function  itself  are 
not  considered.  The  approximations  are  determined  using  a standard  least  squares  fit  in 
terms  of  the  gradients. 

Toropov  and  co-workers  {Maikov  and  Toropov,  1991)  and  Toropov  et  al.,  1993) 
used  mid-range  multi-point  approximations  in  their  optimization  technique.  At  each  cycle 
of  the  optimization  a number  of  new  function  evaluations  is  carried  out.  The 
corresponding  design  points  can  be  viewed  as  a plan  of  experiments  in  the  actual  search 
sub-domain,  with  its  dimension  and  location  controlled  by  a move  limit-strategy.  The 
response  surface  are  fitted  with  function  values  using  a weighted  least  square  technique, 
with  weights  reflecting  the  relative  importance  of  data  to  the  optimization  process.  For 
example,  points  close  to  the  boundary  between  feasible  and  infeasible  domains  are 
weighted  more  heavily  compared  to  the  points  with  severe  constraint  violation.  This 
procedure  attempts  to  achieve  the  best  response  surface  approximations  at  potential 
optimum  regions.  If  design  sensitivities  are  available,  then  it  is  advocated  to  include  this 
information  in  the  response  surface  construction  at  the  condition  that  Design  sensitivities 
should  be  normalized  by  the  norm  of  the  gradient.  The  relative  influence  of  design 
sensitivities  is  controlled  by  a user  defined  non-dimensional  parameter.  In  this 
optimization  strategy,  design  sensitivities  are  included  to  improve  overall  efficiency. 
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There  are  two  aspects  that  may  improve  efficiency.  First,  more  accurate  and 
complex  approximations  might  be  established  if  sensitivities  are  available.  Second,  if 
design  sensitivities  may  be  evaluated  cheaply,  then  the  number  of  design  evaluations  can 
be  reduced  as  additional  data  is  provided  in  terms  of  design  sensitivities.  Toropov  and  co- 
workers compared  the  results  obtained  using  their  method  with  the  results  present  in  the 
literature  for  several  optimization  problems.  For  the  test  problems  the  optimization 
method  including  gradient  information  proposed  by  Toropov  and  co-workers  converged 
to  the  optimum  solution  in  fewer  cycles  then  the  methods  available  in  the  literature.  For 
the  test  problems  the  algorithm  without  gradient  information  was  not  used.  Toropov  and 
co-workers  compared  the  performance  of  their  algorithm  with  and  without  gradient 
information  only  for  one  problem.  It  was  found  that  for  that  specific  problem  the 
algorithm  converged  in  fewer  cycles  when  the  gradient  information  were  not  included  in 
the  approximate  function  formulation.  However  such  a comparison  was  carried  out 
creating  the  same  approximating  function  and  using  the  same  number  of  design  points  for 
both  the  algorithms  with  and  without  gradient  information.  Possibly  the  comparison  is 
somewhat  unfair  since  the  full  potential  of  the  use  of  gradient  information  was  not  used 
since  a higher  order  approximation  could  have  been  generated  at  each  cycle  when 
gradient  information  were  used. 

Derivatives  and  function  values  were  included  in  a regression  procedure  for 
global  approximation  by  Van  Campen  et  al.  (1995)  and  Etman  (1992).  The  derivatives 
and  the  functions  were  combined  along  by  a weighted  least  squares  formulation.  It  was 
assumed  that  the  random  errors  in  the  function  values  and  derivatives  for  a single  point  in 
the  design  space  are  correlated,  and  that  the  covariance  may  vary  form  point  to  point. 
However,  it  was  also  assumed  that  errors  for  different  points  are  uncorrelated.  Weights  in 
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the  regression  were  selected  as  the  inverse  of  the  covariance  matrix.  The  latter  in  not 
known  a priori  and  can  be  approximated  once  an  estimation  of  the  response  surface  is 
available,  leading  to  an  iterative  procedure  for  determination  of  regression  coefficients, 
covariances  and  weights.  Etman  concluded  that  incorporating  derivatives  in  response 
surface  approximation  construction  is  most  beneficial  if  the  approximations  offer 
sufficient  flexibility,  i.e.  higher  order  approximation  functions  with  relatively  large 
number  of  regression  coefficients  are  required,  otherwise  he  recommends  to  use  function 
values  only. 

Wang  and  Grandhi  ( 1 994)  proposed  an  algorithm  for  optimal  design  of  frame 
structures  that  uses  gradient  information.  An  essential  component  of  this  algorithm  is  the 
construction  of  a multivariate  spline  approximation,  based  on  information  gained  during 
the  optimization  process.  The  coefficients  of  the  multivariate  spline  approximations  are 
determined  from  a modified  least  square  formulation.  The  main  motivation  for  including 
derivatives  in  the  least  square  formulation  is  better  accuracy.  For  the  problem  that  the 
authors  investigated  design  sensitivities  could  be  easily  calculated.  A similar  procedure 
was  followed  for  reliability  oriented  optimization  by  Wang  and  Grandhi  (1995).  The 
method  used  by  the  authors  can  be  classified  as  a mid-range  method.  The  main  difference 
between  the  work  of  Wang  and  Grandhi  and  Toropov' s method  is  that  the  latter  allows 
additional  points  to  be  added  in  the  actual  search  sub-domain.  Moreover,  Wang  and 
Grandhi  adapt  the  complexity  of  their  spline  approximations  as  more  information 
becomes  gradually  available.  In  contrast  to  this,  Toropov  and  co-workers  tend  to  use  the 
same  type  of  response  surface  approximation  for  all  cycles  of  the  optimization  process. 

Motivated  by  gradients  that  can  be  evaluated  cheaply,  Dyck  and  Lowther  (1998) 
investigated  response  surface  techniques  using  information  on  both  function  values  and 
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derivatives.  To  be  more  specific,  an  elaboration  on  the  interpolated  moving  least  square 
(IMLS)  formulation  ( Lancaster  and  Salkauskas,  1 986)  has  been  considered.  Dyck  and 
Lowther  applied  their  methodology  to  the  designs  of  loudspeakers  magnet  assembly.  In 
this  formulation  the  weight  allocated  to  each  of  the  sample  points  depends  on  the  distance 
between  the  point  for  which  the  response  surface  approximation  is  being  evaluated  and 
the  sample  points.  The  resulting  set  of  equations  is  solved  using  singular  value 
decomposition.  It  is  suggested  that  weights  selected  for  derivatives  should  be  different 
form  those  attributed  to  function  values. 

Least  square  formulation  have  been  used  extensively  for  the  formulation  of  stress 
recovery  techniques.  These  techniques  are  used  to  construct  stress  distributions  that  have 
better  accuracy  than  the  underlying  finite  element  solution.  Often  the  main  purpose  is  to 
generate  local  error  estimates,  which  can  be  used  as  a basis  for  adaptive  mesh  refinement. 
In  contrast  to  most  stress  recovery  schemes,  Riggs  et  al.  ( 1 997)  also  included  information 
on  derivatives  in  the  least  square  formulation.  In  this  way,  C^-continuos  recovered  stress 
distributions  are  established.  The  weighting  of  derivatives  versus  function  values  was 
done  using  a user-defined  non-dimensional  parameter  and  the  length  of  the  element  sides. 
It  was  reported  that  the  results  are  remarkably  insensitive  to  the  selection  of  the  user- 
defined  parameter.  In  a more  recent  work  by  Tesser  el  al.  (1998)  the  formulation  was 
augmented  with  an  additional  term  to  reduce  the  “curvature”  of  the  recovered  stress 
distribution.  The  purpose  was  to  improve  the  smoothing  capabilities  of  the  response 
surface  approximation. 

However,  current  use  of  gradient  information  in  the  construction  of  response 
surface  approximation  is  still  based  on  the  theory  developed  for  response  surface  not 
based  on  gradient  information.  The  design  of  experiment  techniques  and  the  statistical 


tools  that  are  used  to  reduce  the  number  of  terms  in  a response  surface  while  improving 
the  accuracy  of  the  response  surface  are  not  yet  available  for  response  surface  including 
derivatives.  The  proposed  research  will  address  these  issues. 


CHAPTER  3 

RESPONSE  SURFACE  METHODOLOGY 


Response  Surface  methodology  can  be  defined  as  a collection  of  statistical  and 
mathematical  techniques  useful  for  developing,  improving,  and  optimizing  processes. 

The  most  extensive  application  of  response  surface  methodology  can  be  found  in  the 
industrial  world,  in  situations  where  several  input  variables  influence  some  performance 
measure,  called  the  response,  in  a way  that  is  difficult  or  impossible  to  describe  with  a 
rigorous  mathematical  formulation.  In  these  situations  it  might  be  possible  to  derive  an 
expression  for  the  performance  measure  based  on  the  response  values  obtained  from 
experiments  at  some  particular  combination  of  the  input  variables.  The  expression  of  the 
performance  measure  obtained  through  experiments  is  called  response  surface  (RS).  This 
chapter  introduces  the  basic  mathematical  and  statistical  concepts  that  belong  to  the 
response  surface  methodology.  These  concepts  allow  deriving  a response  surface 
approximation  from  experiment  values,  choose  the  number  and  the  location  of  the 
experiments  in  the  design  space,  and  measure  the  accuracy  of  the  response  surface  once  it 
is  obtained. 

Fitting  an  approximation  to  given  data 

In  this  section  it  is  understood  that  the  experimental  data  are  provided  to  the 

designer  and  that  his  task  is  to  obtain  an  accurate  response  surface  approximation  using 
the  given  data.  For  the  moment  it  is  also  assumed  that  the  designer  is  not  concerned  with 
the  number  and  the  location  of  the  experiments  in  the  design  space. 
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It  is  assumed  that  a relationship  that  describes  the  performance  measure  of  a 
phenomenon  that  it  is  under  investigation  exists,  and  that  this  relationship  is  a function  ol 
some  design  variables.  It  is  important  to  note  that  this  relationship  does  not  have  to  be 
known  explicitly  or  even  be  possible  to  be  known  exactly.  The  response  that  is  to  be 
approximated  is  denoted  as  y,  and  in  principle  it  could  represent  any  measurable  quantity 
such  as  stresses,  chemical  concentration,  crop  production,  national  debt  etc.  The  true 
response  of  the  performance  measure  is  denoted  by  r/  and  it  is  a function  of  the  design 

variable  vector  x of  dimension  ndv,  that  is 

r]  = rj(xx,x2,..,xnJ  = rj{\)  (3.1) 

The  response  is  generally  obtained  from  experiments,  in  which  case  ?]  denotes  the 
mean  or  expected  response  value.  At  any  combination  of  the  design  variables,  the  value 
of  the  experimentally  obtained  y differs  from  the  expected  value  r]  due  to  random 
experimental  error  s with  mean  of  zero.  The  relation  between  y and  the  vector  x of  ndv 
design  variables  can  be  written  as 

y = rj(x,  $)  + £ (3-2) 

where  (3  is  the  vector  of  parameters.  If  a response  surface  approximation  y is  used  to 

approximate  rj  the  previous  equation  can  be  written  as 

y(x)  = rj(x,  b)  (3.3) 

where  b an  estimate  of  (3.  The  choice  of  the  function  q(x,P)  is  very  important  in  order  to 

achieve  a good  approximation.  However  this  is  also  a very  difficult  task  since  the  true 
form  of  q(x,P)  is  not  known,  and  therefore  there  is  usually  no  way  of  knowing  which 

class  of  function  give  the  best  approximation.  The  form  of  the  approximation  ri(x,P) 
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depends  on  shape  functions  and  coefficients  of  the  shape  functions.  For  example  one  of 
the  most  common  forms  of  q(x,P)  is 

7ftP ) = ^fiifi(x)  (3.4) 

/=1 

where  p is  the  number  of  coefficients  and  shape  functions  of  r|(x,P) . For  example,  the 

yield  per  acre  p of  some  farmland  could  be  a function  of  the  average  seasonal 

temperature  T and  the  amount  of  fertilizer  F used  by  a farmer.  Forms  of  rj  that  might 
approximate  the  true  response  as  a function  of  the  design  variables  T and  F in  a form 
similar  to  the  form  of  Eq.  (3.4)  could  be 

V (T,F)  = + fix  T + P2F2  (3.5) 

or  in  a different  form 

n(T,F)  = K+li]T'n+—L-  (3.6) 

f + p2 

Both  Eq.  (3.5)  and  Eq.  (3.6)  have  three  constant  terms  p0,  Pi  and  In  Eq.  (3.5)  p 
is  a linear  function  of  the  coefficients  /?,.  In  contrast  the  last  term  of  Eq.  (3.6)  is 
non-linear  in  P2.  Note  that  the  linearity  or  non-linearity  refers  to  the  coefficients  of  the 
response  surface  and  not  the  shape  functions.  Nonlinear  forms  like  Eq.  (3.6)  lead  to  so 
called  “non-linear  regression  ” and  are  not  used  in  this  work.  The  term  “linear  RS”  is 
confusing  because  for  many  people  it  implies  first  order  polynomials.  What  is  linear  is 
the  regression  not  the  response  surface  form. 

Once  the  form  of  p is  set,  that  is  its  shape  functions  are  chosen,  the  designer  has 
to  find  the  estimates  6,  of  px  using  the  available  data.  The  best  set  of  parameters  refers  to 
the  parameters  that  minimize  some  error  measure.  The  most  popular  of  these  measures  is 
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by  far  the  square  root  of  the  error  mean  square  ( -V MSE  or  RMSE),  where  MSE  is  defined 
by 


1 


n. 


MSE  = y>,-'/(x„P)] 


(3.7) 


n-p  i 


where  n is  the  number  of  experiments  available  to  the  designer  and  p is  the  number  of 
parameters  in  the  response  surface.  Other  error  measures  that  sometimes  are  used  to  find 
the  best  set  of  parameters  estimates  bt  are  the  average  error  (AE) 


AE  = ~YJ\y,  -*7(X/,P) 

n M 


(3.8) 


and  the  maximum  error  (ME) 


ME  = max -^(x,,P) 


/ e n 


(3.9) 


In  this  study,  the  V MSE  or  RMSE  is  used  as  the  measure  of  the  error  in  order  to 
find  the  best  set  of  estimates  bj  to  the  parameters  p,. 

When  the  only  error  present  in  the  experimental  data  is  random,  the  value  of  the 
components  of  the  vector  of  parameters  p that  would  be  found  if  we  had  an  infinite 
number  of  experiments  is  sometimes  thought  as  the  “true”  value  of  the  parameters  of  the 
RS.  However,  with  a finite  number  of  experiments  only  an  estimate  of  P,  herein  denoted 
by  b,  can  be  obtained.  As  a general  rule  of  thumb  the  larger  the  amount  of  data  available 
the  more  accurate  the  estimates  b,  even  though  this  is  not  always  true. 

In  order  to  illustrate  how  to  calculate  the  estimates  b that  minimize  the  RSME 
from  the  experimental  data,  it  is  assumed  that  the  response  surface  approximation  has  p 
coefficients  or  that  the  vector  of  coefficients  /?  has  dimension  p (in  the  yield  example  of 
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Eq.  (3.5)  and  Eq.  (3.6)  p = 3).  Moreover  it  is  assumed  that  there  is  no  modeling  error,  i.e. 
p can  represent  the  true  response  that  is 

y;  =^(xi,p)  + £j  for  / = 1,..,  n (3.10) 

For  the  derivation  of  the  b vector  of  estimates  parameters  in  presence  of  bias  error 
see  Khuri  and  Cornell  ( 1 996). 

Since  the  response  surface  approximation  is  linear  in  the  parameters  /?,  Eq.  (3.4) 
can  be  conveniently  written  in  matrix  form  as 

r]  = Xp  (3.11) 

or 


r ^ 

d\ 

/i(X|)  /2(xi)  •••  4(x ,) 
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/i(x2)  /2(x2)  •••  4U2) 
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• 

(3.12) 

Jin  , 

_/i(x„)  /2(x„)  •••  4(x»)_ 

N 

Ci, 

J 

where  X is  a matrix  of  response  surface  shape  functions  evaluated  at  the  data  points  x„ 
and  the  vector  p contains  the  unknown  coefficients  J3,  of  the  response  surface 
approximation  p.  The  shape  functions  used  in  response  surface  methodology  are  usually 
low  order  polynomials,  and  then  Eq  (3.12)  takes  a simple  form,  since ./i(xj)  are 
monomials. 

For  example,  if  the  response  surface  approximation  is  a first  order  polynomial 
function  as 

H (X>P)  = /?o  + Axi  + Pix 2 + •••  + PpxP  (3-13) 

its  matrix  form  given  by  n design  points  is 
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(3.14) 


V * J 


where  the  first  index  in  the  elements  of  the  X matrix  in  Eq.  (3.14)  refers  to  the  data 
number  and  the  second  refers  to  the  design  variable.  The  difference  between  the  model 
and  the  j experiment,  called  the  residual  is 


r,  = y , - y{xj ) 


(3.15) 


Equation  (3.15)  can  be  written  in  vector  form  so  to  include  all  the  n points  as 


r = y-Xp 


(3.16) 


where  y is  the  vector  whose  elements  are  the  values  of  the  experimental  data.  The 


V MSE  or  RSME  that  follows  from  Eq.  (3.17)  can  be  conveniently  written  as 


4MSE  = RMSE 


1 


T 

r r 


n-  p 


(3.17) 


From  Eq.  (3.17)  is  obvious  that  the  vector  of  estimated  parameters  b that 
minimizes  the  RMSE  is  the  same  one  that  minimizes  r r.  Substituting  Eq.  (3.16)  in  r r we 
obtain 

rTr  = y1  y - yTXp  - pTXy  + p TX  TXp  (3.18) 

T 

The  vector  b that  minimizes  the  RMSE  can  be  found  by  differentiating  r r with 
respect  to  the  components  of  ft  and  set  them  to  zero.  This  leads  to 

XTXb  = XTy  (3.19) 

Equation  (3.19)  reduces  to  a p x p system  of  equations  called  the  normal  equation , 


that  can  be  solved  for  b as 
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b = (XTX)-'XTy  (3.20) 

For  moderate  and  large p,  (X'X)  is  very  ill-conditioned  and  b is  computed  using 
numerical  algorithms  such  as  QR  decomposition  that  solve  directly  the  equation  y = Xb 

for  the  least  square  solution.  Furthermore  it  is  important  to  note  that  in  order  to  improve 
numerical  stability  is  a good  practice  to  scale  all  variables  so  that  each  variable  changes 
in  the  range  (-1,+1). 

If  the  error  e satisfy  the  conditions  of  i)  E(e)  - 0,  ii)  Var(s)  = E(s'£)  = c^In  where 
E(*)  indicates  the  expected  value  and  In  is  the  identity  matrix  of  order  n then  the  model 

t]  = Xp  (3.21) 

is  correct  and  b is  an  unbiased  estimator  of  p ( Khuri  and  Cornell,  1 996) 

£(b)  = p (3.22) 

and 

Var(b)  = (X  rX)~'  cr2  (3.23) 

It  is  important  to  note  that  cf  is  rarely  known  and  that  an  unbiased  estimate  r of 
o2  is  often  used 

cr2  * s2  = MSE  = (3.24) 

n — p 

where  in  Eq.  (3.24)  the  sum  of  the  square  errors  (SSE)  is  the  sum  of  squares  of  errors 
unaccounted  for  by  the  fitted  model  is 

SSE  -yTy  - bTXry  (3.25) 

Once  the  value  of  the  vector  components  b is  obtained,  it  is  possible  to  calculate 
the  variance  of  the  parameter  estimates  bt  from  Eq.  (3.23). 
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Evaluation  of  predictive  capabilities 

Response  surface  approximations  are  often  used  to  replace  constraints  or 
objective  functions  in  optimization  problems.  Optimization  problems  however  have  the 
general  tendency  of  exploiting  weakness  in  the  formulation  of  the  RS.  In  terms  of 
response  surface  approximation,  this  means  that  the  optimizer  tends  to  drive  the  optimum 
design  to  regions  where  the  approximations  are  inaccurate.  Highly  accurate  response 
surface  approximations  are  thus  a requirement  for  structural  optimization  application. 
Various  statistical  tools  are  available  in  order  to  evaluate  the  predictive  capabilities  of  an 
response  surface  approximation  from  the  data  used  to  generate  them.  In  addition,  in  this 
dissertation  the  accuracy  of  response  surface  approximations  is  ultimately  checked  by 
supplementary  sets  of  experiments.  If  the  prediction  capabilities  of  the  response  surface 
are  satisfactory,  the  second  set  of  data  can  then  be  added  to  the  first  to  obtain  a more 
accurate  RS. 

For  the  first  set  of  data,  the  coefficient  of  determination  R , the  adjusted  R 
statistic,  R'Adj,  the  percent  root  mean  square  error  %RMSE,  and  the  predicted  sum  of 
squares  PRESS  are  usually  calculated.  Moreover,  for  each  coefficient  of  the  response 
surface  approximations  the  t-statistic  is  calculated.  The  t-statistics  indicate  the  probability 
of  a coefficient  of  being  zero  rather  than  the  value  calculated  from  the  regression 
procedure.  The  coefficient  of  determination  R ~ is  the  proportion  of  total  variation  of  the 
values  of y/  about  the  mean  y explained  by  the  fitted  model. 


n2  SSR  , SSE 
R~  = = 1 

.S' ST  SSY 


(3.26) 


where,  SSY  is  the  sum  of  squares  of  the  deviations  of  observed  y,  about  their  average 
value  y 
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r (1  V) 

SSY  = yTy-y  y}  (3.27) 

n 

where  the  vector  1 is  a vector  whose  components  are  all  equal  to  the  unity  and  it  has  the 
same  dimension  of  y,  that  is  n.  SSR  is  the  sum  of  squares  of  the  variation  of  y from  the 

same  average  y 


SSR  = bTXTy-V  ^ 

n 


(3.28) 


and  SSE  is  given  in  Eq.  (3.25). 

The  coefficient  of  determination  R ranges  between  1 and  0,  with  1 generally 
indicating  a good  fit  and  0 a bad  one.  Unfortunately,  a value  close  to  1 does  not 
automatically  imply  a good  response  surface  approximation  because  additional 
parameters  added  to  a response  surface  approximation  will  always  increase  the  R2  value 
without  necessarily  increasing  the  predictive  capabilities  of  the  response  surface 
approximation.  The  adjusted  coefficient  of  determination  R Adj  statistic  is  an  alternative 
measure  of  the  explained  variability  that  is  often  used  and  has  the  desirable  property  that 
its  value  does  not  necessarily  increase  when  adding  additional  parameters  to  a response 
surface  approximation.  When  a large  difference  exists  between  R2  and  R2 Adj,  a good 
chance  exists  that  the  response  surface  approximation  contains  insignificant  parameters. 
The  R2 Adj  statistic  is  defined  as 

SSE 

n2  _ 1 / iP 

Adj  SSy/ 

(n 

The  percentage  error  is  defined  in  relation  to  y the  average  absolute  value  of  the 


~P)_ 


= 1- 


-1) 


'(n-l)' 

Kn~P  J 


MO 


(3.29) 


measured  response  data,  where 
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i » 


(3.30) 


so  that  the  %RMSE  is  defined  as 


%RMSE  = 


(3.31) 


where  x = ( n-p ) for  the  first  set  of  data  used  for  the  fit  and  % = n for  the  additional  set  of 


data  used  to  check  the  prediction  capabilities  of  the  RS.  For  the  first  set  of  data  the  (n-p) 
factor  is  used  to  obtain  an  unbiased  estimator  of  the  mean  square  error  of  the  response 
surface  approximation,  compensating  for  the  fact  the  error  is  calculated  at  the  data  points 
used  to  construct  the  RS.  Even  though  Eq.  (3.31)  is  based  on  an  unbiased  estimator  of  the 
mean  square  error,  the  %RMSE  value  may  be  overly  optimistic  in  estimating  the 
predictive  capabilities  of  the  RS.  An  alternative  estimate  of  the  predictive  capabilities  of  a 
response  surface  is  obtained  when  defining  the  %RMSE  based  on  the  PRESS  statistic. 

The  PRESS  statistic  is  calculated  by  excluding  one  data  point,  say  data  point  /.  The 
response  surface  obtained  from  the  remaining  (n-l)  data  points  is  then  used  to  predict  the 
response  at  the  excluded  data  point,  denoted  by  y(i)  The  prediction  error  at  the  excluded 

data  point  may  then  be  written  as 


(3.32) 


and  is  referred  to  as  the  ith  PRESS  residual.  This  procedure  is  then  repeated  for  all 
data  points  of  the  original  data  set  and  the  resulting  PRESS  residuals  are  summed  to 


obtain  the  PRESS  statistic  as  follows 


n 


PRESS  = Yr2 


(3.33) 
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The  zth  PRESS  residual  can  be  calculated  from  the  regression  performed  using  all 
the  n data  points  as  Myers  and  Montgomery  (1995) 


where  h „ denotes  the  diagonal  terms  of  the  matrix  H defined  as 

H = X(XTXr'XT  (3.35) 

However  for  large  problems  Eq.  (3.33)  may  not  be  accurate  enough.  The  %RMSE 
based  on  the  PRESS  statistic  is  then  defined  as 


%RMSE 


PRESS 


100  PRESS 


(3.36) 


TV  ft 

Besides  the  %RMSE  other  error  measures  associated  with  the  second  set  of  points 
are  sometimes  used,  and  these  are  the  percent  average  error  %AE  and  the  percent 
maximum  error  %ME.  These  last  two  quantities  are  defined  as 


%AE  = 


(3.37) 


and 


%ME  = Max 

i e n 


100 

/\ 

\ 

■ 

1 

y 

) 

(3.38) 


Elimination  of  redundant  parameters 

Since  the  exact  form  of  the  actual  response  or  mean  value  //  is  rarely  known, 
polynomial  functions  are  generally  assumed  as  response  surface  approximations.  The 
polynomial  approximation  usually  includes  redundant  parameters,  or  parameters  poorly 
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characterized  by  the  experiments.  These  parameters  might  increase  the  error  in  the 
response  surface  approximation  at  points  other  than  the  data,  thus  decreasing  their 
predictive  capabilities.  Eliminating  these  parameters  from  the  response  surface 
approximation  is  an  important  step  in  constructing  accurate  response  surface 
approximations. 

Several  methods  exist  for  selecting  the  best  subset  of  parameters  from  a general 
polynomial  function  to  be  used  as  a final  response  surface  approximation.  Examples 
include  the  “all-possible-regression"  and  the  “stepwise  regression"  procedures  ( Myers 
and  Montgomery,  1995).  The  all-possible-regression  procedure  requires  that  all  possible 
combination  of  parameters  from  the  general  response  surface  approximation  be 
examined.  The  best  subset  of  parameters  is  then  chosen  based  on  a suitable  criterion.  The 
procedure  requires  the  examination  of  2P  partial  response  surface  approximations  for  p 
parameters  in  the  general  response  surface  approximation. 

The  stepwise  regression  procedure  was  developed  to  provide  an  alternative  to  the 
high  computational  cost  associated  with  the  all-possible-regression  procedure,  especially 
for  response  surface  approximations  with  a large  number  of  parameters.  The  stepwise 
regression  procedure  represents  a family  of  procedures  that  examine  only  a small  number 
of  partial  response  surface  approximations  by  either  adding  or  deleting  one  parameter  at  a 
time.  There  are  forward,  backward,  and  mixed  stepwise  regression  procedures.  The 
forward  stepwise  procedure  assumes  a response  surface  approximation  with  only  one 
parameter,  the  intercept  or  zero  order  term.  Parameters  are  then  added  one  at  the  time  by 
selecting  the  parameter  that  results  in  the  largest  increase  in  the  predictive  capabilities  of 
the  current  response  surface  approximation.  This  process  of  adding  parameters  is 
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repeated  until  a specified  termination  criterion  is  satisfied.  In  contrast,  the  backward 
stepwise  regression  procedure  starts  with  a response  surface  approximation  that  includes 
all  the  parameters  of  the  general  response  surface  approximation.  From  this  response 
surface  approximation,  one  parameter  at  a time  is  deleted,  selecting  the  parameter  with  a 
minimum  influence  on  the  predictive  capabilities  of  the  resulting  response  surface 
approximation.  Finally,  the  mixed  stepwise  regression  procedure  consists  of  a 
combination  of  both  the  forward  and  the  backward  stepwise  procedures.  In  particular  this 
procedure  usually  starts  with  an  initial  response  surface  approximation  that  includes  all 
the  parameters  of  the  general  response  surface  approximation  and  then  applies  backward 
stepwise  regression.  At  each  step  of  the  backward  stepwise  regression,  the  least 
significant  parameter  is  removed  form  the  current  response  surface  approximation. 
Additionally,  at  each  step  all  removed  parameters  are  re-examined  for  possible  inclusion 
in  the  response  surface  approximation.  The  criterion  used  to  determine  when  to 
re-include  a parameter  is  usually  more  stringent  than  the  criterion  used  to  remove  a 
parameter. 

It  is  important  to  underline  that  stepwise  regression  procedures  do  not  guarantee 
the  best  subset  of  parameters  and  must  be  used  with  caution.  However  since  in  the  JMP 
program  SAS  Institute  (1998)  used  in  this  study,  the  three  types  of  stepwise  regression  are 
available,  all  three  were  used.  In  general  good  results  were  obtained  by  applying  first  the 
forward,  then  the  backward  and  finally  the  mixed  stepwise  regression  and  checking  that 
the  three  methods  converged  to  the  same  form  of  the  RS.  When  the  three  methods  did  not 
lead  to  the  same  result,  the  response  surface  with  the  best  termination  criterion  was 


chosen. 
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In  the  present  study.  Mallow’s  Cp  statistic  was  used  as  the  termination  criterion  to 
identify  the  best  reduced  response  surface  approximation  from  the  subset  of  reduced 
response  surface  approximations  provided  by  the  stepwise  regression  procedure.  The  C'p 
statistic  addresses  the  prediction  capabilities  of  the  response  surface  approximation  by 
estimating  a total  error.  The  total  error  consists  of  two  parts,  one  resulting  from  modeling 
errors  (bias)  and  the  other  from  noise  (variance  error).  An  under  specified  response 
surface  approximation  (not  enough  parameters  and  shape  functions)  increases  the  part  of 
the  total  error  resulting  from  bias,  while  an  over  specified  response  surface 
approximation  (too  many  parameters  and  shape  functions)  increases  the  part  of  the  total 
error  resulting  from  variance.  The  Cp  statistic  is  defined  as 


C 


SSE, 
£_ 

MSE 


\ 


in -2  p) 


(3.39) 


J 


where  SSEP  is  the  sum  of  squares  of  the  n errors  for  a response  surface  approximation 
with  p parameters  (including  the  intercept  or  constant  parameter)  and  MSE  is  the  error 
mean  square  of  the  errors  terms  obtained  from  the  response  surface  approximation  with 
all  parameters  included  as. 

MSE  = — — - (3.40) 

n-  p 


Based  on  the  Cp  statistics,  the  best  response  surface  approximation  in  a set  of 
candidate  response  surface  approximations  is  the  response  surface  with  the  lowest  Cp 
value,  and  a good  response  surface  approximation  should  at  least  have  a Cp  value  close  to 


P- 

Another  method  to  discard  unnecessary  parameter  and  so  enhance  the  prediction 
capabilities  of  a response  surface  approximation  relies  on  the  R2 Adj  and  the  student 
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t-statistic  test  denoted  tslat.  In  general  tests  of  hypothesis  concerning  the  individual 
parameters  in  the  proposed  model  are  performed  by  comparing  the  parameter  estimates  b , 
in  the  fitted  model  the  their  respective  estimated  standard  error  ESE(b,).  It  can  be  shown 
from  Eq.  (3.23)  Myers  and  Montgomery  (1995)  that 

ESE(b,)  = ^Var(b),,  = JmSE  pCXf!  (3.41) 

so  that  tslai  is  defined  as 


b, 

ESE(b,) 


(3.42) 


The  student  test  statistic  tstat  gives  an  idea  of  how  reliable  the  estimate  of  a 
parameter  is.  The  higher  the  value  of  tstat  the  more  reliable  is  the  estimate. 


Design  of  experiments 

The  previous  sections  dealt  with  the  issue  of  constructing  a response  surface 
approximation  for  a given  set  of  data.  The  predictive  capabilities  of  response  surface 
approximations  are  strongly  dependent  on  the  distribution  of  design  points  in  the  design 
space.  Therefore,  rather  than  just  choosing  designs  at  random,  it  is  important  to  apply 
some  sound  strategy  for  selecting  a good  set  of  points  at  which  to  carry  out  experiments. 
The  discipline  concerned  with  design  or  planning  of  experiments  is  known  as  design  of 
experiments  (DOE),  and  a set  of  selected  design  points  is  referred  to  as  an  experimental 
design. 

The  most  extensive  body  of  DOE  work  has  been  occupied  with  the  selection  of 
experimental  design  is  regular  design  domain,  i.e.  “box-like”  or  cuboidal  domains,  where 
variables  are  bounded  by  simple  upper  and  lower  limits.  This  has  resulted  in  various 
standard  layouts  for  experimental  design,  which  are  often  preferred  by  experimenters  due 
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to  their  simplicity.  However,  when  the  objective  is  to  build  a response  surface 
approximation  for  design  optimization,  the  feasible  domain  will  not  typically  be  a regular 
one,  since  constraints  tend  to  produce  odd  shaped  design  domains.  It  is  important  to  note 
that  even  though  the  constraints  break  the  regularity  of  the  design  domain  they  serve  the 
great  purpose  of  considerably  reducing  the  size  of  the  design  domain  and  therefore 
helping  to  obtain  more  accurate  response  surface  approximations  Giunta  et  al.  (1994). 

Factorial  designs 

In  box-like  design  spaces  the  range  of  a design  variable  between  the  upper  and 
lower  limit  is  typically  divided  into  levels  and  experiments  are  run  only  at  the  particular 
level  s of  the  design  variables.  A factorial  design  contains  all  the  combination  of  the 
different  levels  of  all  the  design  variables  also  commonly  called  factors.  It  is  easy  to 
understand  how  the  factorial  design  is  not  affordable  for  problems  that  have  more  than  a 
few  design  variables  and  few  levels:  the  number  of  experiments  simply  becomes  too 

large.  The  number  of  experiments  required  by  a factorial  design  is  (/)”*  where  ridv  is  the 

number  of  design  variables  and  / is  the  number  of  levels  per  design  variable.  For  example 
a factorial  design  of  7 design  variables  in  3 levels  has  2187  experiments.  This  is  usually 
too  many  to  be  run.  However  response  surface  approximations  obtained  from  factorial 
design  are  generally  accurate  because  they  avoid  extrapolation. 

There  are  several  special  cases  of  the  factorial  design  that  are  important  to  discuss 
because  they  are  widely  used  and  because  they  form  the  basis  of  other  designs  of 

considerable  practical  value.  The  most  important  of  these  is  perhaps  the  2n,k  factorial 
design.  Figure  7 shows  a two  level  factorial  design  in  three  design  variables  that  is  a 2J 
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factorial  design.  Note  that  in  the  figure  location  of  experiments  are  denoted  by  a black 
dots. 


3 

Figure  7:  Full  factorial  design,  2 

This  design  is  particularly  useful  in  the  early  stages  of  experimental  work  when 
only  the  different  impact  of  the  design  variables  on  the  results  is  of  interest.  The  design 
points  of  a 2",k  factorial  design  in  three-dimensional  space,  i.e.  a problem  with  only  three 
design  variables,  are  the  vertices  of  a cube  as  shown  in  Figure  7.  Because  there  are  only 
two  levels  per  design  variable  only  a linear  response  surface  or  at  most  the  mixed  terms 

of  a quadratic  response  surface  approximation  can  be  obtained  from  the  2",h  factorial 
design. 

The  2n,k  factorial  design  has  the  very  attractive  properties  of  orthogonality  and 
rotatability.  Orthogonality  and  rotatability  can  be  mathematically  defined  based  on  the 
values  of  the  moments  of  the  experimental  design.  Khuri  and  Cornell  (1996)  devoted 
great  attention  to  these  two  concepts  and  the  reader  can  refer  to  that  text  book.  For  the 
moment  is  enough  to  say  that  an  orthogonal  design  is  one  such  that  the  (X  X)  design 
matrix  in  the  normal  equation  Eq.  (3.19)  is  diagonal  which  implies  that  each  parameter  b, 
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in  Eq.  (3.19)  can  be  estimated  independently.  A rotatable  design  instead  is  one  such  that 
the  prediction  variance  v(x)  which  is  given  by 

v(x)  = a2[fT(x)  (XTX)-'  /(x)]  (3.43) 

and  it  depends  only  the  distance  of  x from  the  center  of  the  design  space. 

One  of  the  drawbacks  of  the  2 factorial  design  is  that  it  must  be  assumed  that 
the  response  surface  approximation  is  linear  or  at  most  inclusive  of  the  mixed  terms.  This 

limitation  inhibits  the  use  of  2n,k  factorial  designs  for  many  problems.  When  the  true 
function  that  is  being  approximated  by  the  response  surface  approximation  is  expected  to 

of  have  considerable  curvature,  designers  often  turn  to  3”*  factorial  design.  The 
3",A  factorial  designs  allow  to  estimate  all  the  terms  of  a quadratic  response  surface 

approximation.  An  example  of  a 3n*  factorial  design  is  given  in  Figure  8 for  a three 
variable  case.  The  3J  design  shown  in  Figure  8 is  orthogonal  but  it  is  not  rotatable. 


Fractional  factorial  designs 

Factorial  designs  unfortunately  are  affected  by  the  curse  of  dimensionality,  and 
cannot  be  used  for  high  dimensions.  As  an  example  consider  the  case  of  a problem  where 
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the  behavior  of  a physical  quantity  is  described  by  1 0 design  variables.  Even  the 

2"*  factorial  design  could  hardly  be  used  as  1 024  designs  would  be  required  and  1 024 
experiments  are  too  many  for  most  problems.  Also  note  that  a linear  response  surface  in 
1 0 design  variables  has  only  1 1 parameters  that  have  to  be  estimated  and  therefore  in 
principle  1 1 experimental  design  points  would  be  enough  to  solve  the  normal  equation. 

These  considerations  are  even  truer  for  3"*  factorial  designs.  A three  levels  factorial 
design  in  just  seven  design  variables  consists  of  2,187  experiments,  which  is  enough  to 
rule  out  most  practical  applications.  In  situations  where  the  number  of  design  variables  or 
the  number  of  levels  for  design  variables  is  too  high  designers  usually  rely  on  using  only 
a fraction  of  the  design  points  that  constitute  the  factorial  designs. 

Designs  that  use  only  a fraction  of  the  factorial  design  points  are  called  fractional 

factorial  design.  A fraction  of  a 2"'A'  design  consisting  of  only  2"*’"' treatments 
combination  where  m is  a non-negative  integer  such  that  2n,h,~"'>  n<jv+\  points  is  called 
2“"'  fraction  of  a 2lljv  factorial  design.  For  example,  a 2“'  fractional  design  contains  one- 

half  the  number  of  points  of  a 2"*  design.  The  same  notation  and  definitions  are  used  for 
fractional  factorial  design  with  more  than  two  levels.  Figure  9 shows  a 3(J‘2>  fractional 
factorial  design. 
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Figure  9:  Fractional  factorial  design,  3<3‘7) 


Fractional  factorial  designs  were  first  introduced  by  Finney  (1945).  The  fraction 
factorial  design  used  for  the  estimation  of  the  parameters  in  the  normal  equation  Eq. 
(3.19)  can  be  chosen  is  such  a way  that  it  retains  some  of  the  properties  of  the  full 
factorial  design,  i.e.  orthogonaliy  and  rotatability.  It  has  to  be  pointed  out  though  that 
whenever  we  rely  on  a fractional  factorial  design  we  lose  the  desirable  and  important 
property  of  interpolation  and  unavoidably  for  some  portions  of  the  design  domain  the 
response  surface  approximation  will  predict  based  on  extrapolation  which  usually  leads 
to  poorer  accuracy. 


Central  Composite  Designs 

The  central  composite  design  (CCD)  combines  full  or  fractional  factorial  design 
with  additional  points  to  allow  fitting  of  full  quadratic  polynomials  and  it  was  introduced 
by  Box  and  Wilson  ( Box  and  Wilson , 1951).  It  has  become  since  probably  the  most 
popular  class  of  experimental  designs  for  second-order  models.  The  CCD  is  an 
augmentation  of  the  two-level  full  factorial  design,  with  nc  replication  of  the  nominal 


design  (center-point  of  domain)  and  the  2«</v  points  obtained  by  changing  one  design 
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variable  at  a time  by  a value  of  ± a.  The  most  used  values  of  a are  either  1 .0  or 

XjF  where  F is  the  number  of  factorial  points.  The  later  value  of  a one  is  used  in 
situation  where  rotatability  is  desirable.  Figure  10  shows  an  example  of  CCD  for  three 
dimensional  problem. 
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Figure  10:  Central  composite  design  in  3 design  variables 


The  nominal  design  replication  serve  to  improve  stability,  i.e  to  make  the  model 
less  sensitive  to  errors  in  data  throughout  the  domain.  Of  course  when  the  experiments 
involved  are  numerical  simulations,  as  it  is  in  the  case  of  this  work,  there  is  little  sense  in 
repeating  the  computation  several  times  with  the  same  input  values,  as  the  computer  will 
always  give  the  same  result.  Instead  it  may  be  useful  to  repeat  the  simulations  at  slightly 
perturbed  values.  For  quadratic  response  models,  CCD  is  an  attractive  alternative  to  the 
full  factorial  three-level  design,  since  its  performance  is  nearly  as  good  at  a much  lower 

cost.  In  fact,  the  number  of  points  required  totals  ( 2"*  +2ndv  +nc),  which  is  comparable  to 
that  of  full  two-level  factorials.  For  example,  a CCD  with  seven  variables  and  no  center 
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replications  (nc  = 1)  contains  143  sampling  points,  which  is  substantially  more  tractable 
than  the  2187  run  required  in  the  corresponding  three-level  factorial.  It  is  worth  noting 
that  some  CCD  are  based  on  fractional  factorial  vertices  to  that  the  total  amount  of  design 

points  needed  are  ( 2''*""'  +2ncjv  +nc). 

Plackett-Burman  designs 

The  Plackett-Burman  (PB)  {Placket!  and  Burman,  1 946)  is  yet  an  other  design  for 
first-order  response  surface  approximations.  This  is  a two-level  factorial  design  in  n<iv 
variables  where  the  number  of  design  points  n is  equal  to  n^v  +1 . These  designs  are 
available  only  when  n is  a multiple  of  4.  The  objective  of  this  design  is  to  obtain  designs 
that  can  estimate  all  main  effects  with  maximum  precision  possible  for  n = nc/v+\. 

To  construct  a Plackett-Burman  design  in  n,jv  variables,  a first  row  consisting  of 
elements  equal  to  +1  and  -1  has  to  be  selected  such  that  the  number  of  positive  ones  is 

yj  -f  1 yi  — ] 

— and  the  number  of  negative  ones  is  — — — . Manipulating  the  first  row  cyclically 

the  PB  design  can  be  obtained  for  any  problem  that  satisfies  the  condition  that  the 
number  of  design  variables  is  a multiple  of  four.  Fortunately  most  statistical  programs  as 
JMP  can  generate  the  Plackett-Burman  design  automatically  for  the  user. 

Box-Behnken  design 

The  Box-Behnken  design  (BBD)  Box  and  Behnken  (1960)  is  a three-level 
incomplete  factorial  design  for  the  estimation  of  the  parameters  in  a second-order  model. 
By  definition,  a three-level  incomplete  factorial  design  is  a subset  of  the  factorial 

combinations  from  the  3''*  factorial  design.  The  particular  arrangement  of  the  BBD 
allows  the  number  of  design  points  to  increase  at  the  same  rate  as  the  number  of 
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polynomial  coefficients.  The  BBD  are  found  by  perturbing  combinations  of  two  variables 
in  turn,  and  adding  to  this  nc  replicates  of  the  nominal  design.  When  the  experiments  are 
numerical  in  nature,  instead  of  repeating  the  nominal  design  experiment  slight 
perturbations  are  needed.  The  two  variables  are  varied,  from  their  nominal  value  to  a low 
and  high  level  respectively,  in  all  the  four  possible  ways.  Thus,  since  there  are 


'CvKv+1) 

2 


ways  to  select  two  variables  out  of  n, ^ ones  a BBD  consists  of  totally 


2 ndAndv  ~ 1)  + nc  + 1 points.  For  large  number  of  variables,  the  amount  of  points 

asymptotically  approaches  four  times  the  number  of  terms  to  fit  in  a quadratic 
polynomial.  A seven  variable  BBD  with  no  center-point  replication  holds  85  designs, 
nearly  half  of  the  143  runs  required  by  the  CCD  and  only  a fraction  of  the  2187  needed 
for  a full  3 level  factorial  design.  An  example  of  BBD  for  three  design  variables  is  shown 
in  Figure  1 1 


X3  A 


Figure  1 1 : Box-Benhken  Design 


The  inevitable  drawback  of  a BBD,  as  it  can  be  seen  from  Fig  1 .5  is  that 
corner-regions  of  the  domain  are  poorly  represented.  Ideally  speaking,  as  already 
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mentioned  previously,  response  surface  approximations  should  only  be  used  for 
predicting  responses  from  interpolation  and  not  extrapolation. 


Simplex  design 

The  simplex  design  is  an  orthogonal  design  consisting  of  n = njv  +1  points  where 
njv  is  the  number  of  variables  in  the  first-order  model.  Unlike  the  PB  design,  which  has 
the  same  number  of  design  points,  the  simplex  can  be  used  for  any  number  of  design 
variables.  The  design  points  of  the  simplex  design  are  located  at  the  vertices  of  a «^v- 
dimensional  regular  sided  figure,  or  a simplex,  and  are  characterized  by  the  fact  that  the 

angle,  0,  which  any  two  points  make  with  the  origin  is  such  that  cos(0)  = — — ( Khuri 

”dv 

and  Cornell,  1996).  The  simplex  is  the  design  that  requires  the  minimum  amount  of 
points  to  evaluate  a first-order  RS.  A simplex  design  in  three  dimensions  is  shown  in 
Figure  12. 


x3* 


X 
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Figure  12:  Simplex  design 
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However  the  main  drawbacks  of  this  design  is  that  as  the  number  of  design 
variables  increases  the  volume  of  the  simplex  increases  exponentially  therefore  affecting 
negatively  on  the  accuracy  of  the  RS.  Moreover  the  simplex  might  have  just  too  few 
design  points  to  obtain  a meaningful  response  surface  approximation.  The  number  of 
design  points  of  the  simplex  design  is  barely  enough  to  perform  a fit  so  that  there  is  no 
information  left  to  estimate  errors  and  accuracy  of  the  response  surface  obtained  utilizing 
this  design. 

D-Optimalitv 

The  experimental  designs  that  were  presented  in  the  previous  sections  were  all 
based  on  the  assumption  that  the  experimental  design  space  is  a regular  space  design, 
either  box-like  or  a simplex.  However  it  is  not  uncommon  for  the  constraints  bounding 
the  design  space  to  create  a non-regular  type  of  design  space.  As  mentioned  before  it  is 
generally  advisable  to  invoke  as  many  constraints  as  possible  to  reduce  the  size  of  the 
feasible  design  space,  because  this  will  typically  improve  the  response  surface  accuracy. 
Design  optimality  criteria  are  usually  used  for  non-regular  design  spaces  and  became 
popular  because  they  are  implemented  in  all  major  statistical  packages  of  which  JMP  is 
no  exception.  The  methods  available  to  generate,  evaluate,  and  compare  different  design 
were  the  results  of  the  work  of  Kiefer  ( Kiefer , 1959  and  Kiefer,  1961 ) and  Kiefer  and 
Wolfowitz  ( Kiefer  and  Wolfowitz,  1959).  With  the  increasing  popularity  of  computers, 
algorithms  were  developed  to  choose  the  best  design  by  a software  package  such  as  JMP, 
based  on  the  practitioner’s  choice  of  sampling  size,  model,  ranges  on  variables  and  other 
constraints.  The  most  common  of  the  optimality  conditions  are:  D-Optimality, 
A-Optimality  and  G-optimality  often  referred  to  as  alphabetic  optimality  criteria. 
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D-Optimality  is  based  on  the  moment  matrix  M 

M _ (3.44) 

N 

The  inverse  of  the  moment  matrix,  M " , as  can  be  seen  from  Eq.  (3.23),  contains 
the  variances  and  covariances  of  the  regression  coefficients,  scaled  by  N/cr.  As  a result, 
control  of  the  moment  matrix  implies  control  of  the  variances  and  covariances. 

The  determinant 


XTX 

N~p 


(3.45) 


is  inversely  proportional  to  the  square  of  the  volume  of  the  confidence  region  on  the 
regression  coefficients  under  the  assumption  of  independent  normal  errors  with  constant 
variance.  The  volume  of  the  confidence  region  reflects  how  well  the  set  of  coefficients 

T T 1 

are  estimated.  A small  |X  X|  and  hence  a large  |(X  X)  | implies  a poor  estimate  of  the  p 
vector  of  coefficients  in  the  model. 

A D-optimal  design  is  one  in  which  the  relationship  of  Eq.  (3.42)  is  maximized; 

that  is 


Max  M(0 


(3.46) 


where  Max  implies  that  the  maximum  is  taken  over  all  designs.  As  a result,  it  is  possible 
to  define  the  D-Efficiency  of  a particular  design  C,*  as 


A#  ~ 


f i , | V'/7 

me ) 


Max  |M(0| 


(3.47) 


J 


The  1/p  power  accounts  for  the  p parameter  estimates  being  assessed  when  one 


• T 1 

computes  the  determinant  of  the  scaled  variance-covariance  matrix  (X  X)"  . The 
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definition  of  D-efficiency  is  particularly  useful  because  in  principle  it  allows  for 
comparing  designs  that  have  different  sample  sizes  and  it  can  compare  different  designs 
in  design  regions  that  are  not  regular  or  box-type.  Besides  the  advantages  just  mentioned, 
the  D-optimality  does  not  restrict  the  number  of  experiments  to  be  made.  Finally,  this 
variance  based  criteria  takes  full  account  of  the  approximation  model,  which  must  be 
specified  prior  to  the  selection  of  the  sampling  points,  in  contrast  with  the  standard 
experimental  designs.  For  instance,  the  D-Optimal  design  in  a given  design  domain  will 
not  be  identical  for  a first  and  a second  order  response  surface  approximation.  D-Optimal 
designs  tend  to  select  design  points  at  the  boundary  of  the  design  domain  so  to  increase 
the  design  moment  M. 

In  principle,  the  construction  of  D-optimal  designs  is  an  optimization  problem, 
which  is  generally  rather  difficult  to  handle  due  to  the  large  number  of  involved 

T 

variables,  (n  x n rfv)  and  the  complicated  nature  of  the  objective  function  |X  X|,  and 
because  of  the  existence  of  several  local  minima. 

Special  algorithms  that  treat  the  problems  as  combinatorial  ones  have  been 
developed  but  the  resulting  integer  programming  is  still  a very  large  one.  Solutions 
algorithms  using  heuristic  techniques  must  usually  settle  for  suboptimal  (yet  very  good) 
sets  of  designs.  For  example,  there  are  approximately  1010  possible  ways  to  pick  10 
points  out  of  50  candidates.  A popular  way  to  seek  D-Optimal  designs  are  exchange 
algorithms,  which  successively  remove  and  add  points  starting  from  an  initial  arbitrarily 
chosen  design  ( Fedrov , 1 972  and  Mitchell,  1 974).  Also,  genetic  algorithms  have  been 
successfully  applied  for  determining  D-optimal  designs,  see  for  example  Giunta  et  al. 
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(1994).  In  this  work  the  DETMAX  algorithm  first  presented  by  Mitchell  (1974)  as 
implemented  in  the  commercial  statistical  package  JMP  has  been  used. 

Latin  Hypercube  Sampling 

The  latin  hypercube  sampling  (LHS)  is  yet  an  other  design  scheme  that  was 
successfully  used  to  select  design  points  where  to  perform  experiments  to  construct 
response  surface  approximation.  The  LHS  scheme  presented  here  follows  the 
explanations  given  by  McKay  et  al.  (1979)  in  their  study. 

Consider  a quantity  y that  is  a function  of  the  x , variables  with  i =1  We  are 
interested  in  constructing  an  approximation  to  the  function  y using  n numerical 
experiment  obtained  from  n design  points.  The  LHS  scheme  can  be  used  to  select  the  n 
designs  points  where  to  conduct  the  n numerical  experiments.  According  to  the  LHS 
scheme,  the  range  of  each  design  variable  is  divided  into  n nonoverlapping  intervals  on 
the  basis  of  equal  probability.  One  value  from  each  interval  is  then  selected  at  random 
with  respect  to  the  probability  density  in  the  interval.  The  n values  thus  obtained  for  xj 
are  paired  in  a random  manner  with  the  n values  of  xi.  These  n pairs  are  combined  in  a 
random  manner  with  the  n values  of  xj  to  form  n triplets,  and  so  on,  until  n ^-triplets  are 
formed.  In  these  way  n design  points  are  generated  where  the  function  y can  be 
evaluated. 

Consider  a simple  example  where  it  is  desired  to  generate  a LHS  of  size  n = 5 
with  two  input  variables.  Let  us  assume  that  the  first  random  variable  x/  has  a normal 
distribution  with  a mean  value  /u  and  variance  u~.  The  five  intervals  of  the  variable  xj  are 


shown  in  Figure  13. 
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Figure  13:  Intervals  used  with  a latin  hypercube  sample  (LF1S)  of  size  n - 5 in  terms  of 

the  density  function  for  the  normal  random  variable  xy 

The  intervals  of  Figure  13  satisfy  the  condition  that  each  interval  has  the  same 
probability,  P(-  oo  < x,  < A ) = P(  A < x,  < B ) =. . .=  0.2. 

It  is  also  assumed  for  this  example  that  x?  has  a uniform  distribution  on  the 
interval  G to  L as  shown  in  Figure  14.  The  corresponding  intervals  used  in  the  LHS  forx2 
are  given  in  Figure  14. 
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Figure  14:  Intervals  used  with  a latin  hypercube  sample  (LHS)  of  size  n - 5 in  terms  of 

the  density  function  for  the  uniform  random  variable  X2 
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The  next  step  in  obtaining  the  LHS  is  to  pick  specific  values  x/  and  %2  in  each  of 
their  five  respective  intervals.  This  selection  must  be  done  in  a random  manner  with 
respect  to  the  density  in  each  interval;  that  is,  the  selection  must  reflect  the  height  of  the 
density  across  the  interval.  For  example  in  Figure  13  in  the  (-00, A)  interval  forx/,  values 
close  to  A will  have  a higher  probability  of  selection  than  those  values  in  the  tail  of  the 
distribution  that  extends  to  -00 . Next  the  selected  values  of  x/  and  X2  are  randomly  paired 
to  form  the  five  required  two-dimensional  input  vectors.  For  example  two  random 
permutations  of  the  integers  (1,2, 3, 4, 5)  are: 

• Permutation  Set  1:  (3, 1,5, 2,4) 

• Permutation  Set  2:  (2,4, 1,3, 5) 

By  using  the  respective  position  within  these  permutation  sets  as  interval  numbers 
for  x/  (Set  1 ) and  x2  (Set  2)  it  is  possible  to  pair  the  two  design  variables  levels.  Thus  for 
the  first  of  n computer  runs,  the  input  vector  is  formed  by  selecting  the  specific  value  of 
x/  from  the  interval  number  3 (B  to  C in  Figure  13)  and  pairing  this  value  with  the 
specific  value  ofx^  selected  from  interval  number  2 (H  to  I in  Figure  14).  The  vectors  for 
the  second  and  subsequent  runs  are  then  constructed  in  a similar  manner.  Once  the 
specific  values  of  each  variable  are  obtained  to  form  the  five  input  vectors,  a two- 
dimensional  representation  of  the  LHS  such  as  that  given  in  Figure  15  can  be  made. 
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Figure  15:  Two  dimensional  representation  of  one  possible  latin  hypercube  sample  (LHS) 
of  size  n = 5 using  one  normally  distributed  design  variable  x/,  and  one  uniformly 

distributed  design  variable  x? 


Note  that  in  Figure  15  all  of  the  intervals  forx/  have  been  sampled,  and  the  same 
is  true  of  xj.  In  general,  a set  of  n LHS  points  in  ^-dimensional  space  contains  one  point 
in  each  of  the  intervals  for  each  of  the  k variables.  This  feature  is  particularly  important 
when  fitting  higher  than  second  order  polynomial  response  surface  approximations. 

If  for  example  a quartic  polynomial  in  two  design  variables  is  desired  at  least  1 5 
design  points  are  needed  since  a quartic  polynomial  has  1 5 coefficients.  However  the  1 5 
design  points  must  have  at  least  5 different  levels  for  each  design  variables.  With  less 
than  five  levels  the  coefficients  of  the  fourth  power  can  not  be  estimated.  The  LHS 
guarantees  that  there  are  a sufficient  number  of  levels  to  fit  the  quartic  polynomial  if  1 5 
points  are  chosen. 
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Other  methods  for  obtaining  designs 

There  are  several  other  properties  of  the  moment  matrix  M that  have  been 
proposed  as  criteria  for  the  purpose  of  constructing  optimal  experimental  designs.  A- 
Optimality  is  a criteria  that  seeks  to  minimize  the  trace  of  the  moment  matrix  M.  An 
other  method  that  has  been  proposed  as  a valid  criterion  to  evaluate  the  design  efficiency 
is  the  G-optimality.  The  G-Optimality  seeks  to  minimize  the  maximum  prediction 
variance  over  the  experimental  region 

v(x)  = o-2[/r(x)(XTX)-'  /(x)]  (3.48) 

where  /(x)  is  the  function  containing  the  monomials  or  shape  functions  of  the  response 
surface  approximation  form. 

All  the  experimental  designs  previously  considered  are  based  on  the  assumption 
that  the  form  of  the  response  surface  approximation  is  correct  and  therefore  they  are 
concerned  only  with  reducing  the  influence  of  the  noise  in  the  data  or  in  other  words  they 
are  minimum  variance  designs. 

Another  class  of  designs  is  primarily  concerned  with  modeling  or  bias  errors,  and 
therefore  they  try  to  space  the  design  points  in  the  design  domain  so  to  minimize  the  bias 
error.  One  drawback  of  minimum  bias  designs  is  that  in  order  to  choose  the  appropriate 
design  that  minimizes  the  bias  error,  the  true  function  has  to  be  known,  a situation  that  is 
rarely  encountered.  In  general  since  polynomials  are  usually  used  by  designers  as 
approximating  functions  it  is  assumed  that  the  true  function  is  a degree  higher  than  the 
polynomial  used  as  response  surface  approximation.  For  example  it  is  possible  to  find 
design  points  that  minimize  the  bias  error  using  for  example  a quadratic  response  surface 
approximation  when  the  true  function  is  a cubic  polynomial.  Box  and  Draper  (1987), 
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Myers  and  Montgomery  (1995)  and  Khuri  and  Cornell  ( 1 996)  discuss  minimum  bias 
design  in  some  detail. 

Optimization 

One  of  the  most  attractive  features  of  using  response  surface  methodology  in 
optimization  is  that  due  to  the  least  square  procedure  and  smooth  shape  functions,  the 
resulting  approximate  response  function  is  smooth.  Numerical  noise  in  simulations 
creates  spurious  local  minima  that  can  easily  trap  the  optimization  algorithm  and  make  it 
difficult  to  find  the  best  design  Giunta  et  al.  ( 1 994)  and  Venter  et  al.  (1996). 

When  the  noisy  response  function  obtained  from  numerical  simulation  is  replaced 
by  a smooth  response  surface  approximation  one  effectively  filters  out  the  numerical 
noise,  thus  allowing  the  use  of  derivative  based  optimization  algorithms.  Derivative 
based  algorithms  are  efficient  algorithms  and  are  readily  available  commercially.  In  this 
dissertation,  when  not  stated  otherwise,  the  generalized  reduced  gradient  algorithm 
provided  with  the  EXCEL  (. Microsoft  Corporation,  1998)  program  was  used  to  solve  the 
optimization  problems.  The  Excel  program  provided  an  easy  to  use  and  universally 
available  interactive  environment  for  the  solution  of  optimization  problems  and  for  this 
reason  it  was  preferred  to  other  software. 


CHAPTER  4 

STRUCTURAL  OPTIMIZATION  OF  A HAT  STIFFENER  PANEL 


Summary 

This  chapter  describes  a design  study  for  structural  optimization  of  a hat- 
stiffened  laminated  composite  panel  concept  for  an  upper  cover  panel  ot  a typical 
passenger  bay  of  a blended- wing-body  (BWB)  type  transport  airplane.  The  feasibility  of 
a hat-stiffened  composite  panel  concept  for  an  upper  cover  panel  is  investigated,  and  is 
compared  on  the  basis  of  weight,  to  a thick  sandwich  concept,  which  has  been  selected  by 
designers  as  the  baseline  concept  for  the  upper  cover  panel.  The  upper  cover  panel 
concept  is  designed  for  two  load  cases,  internal  pressure  only,  and  combined  internal 
pressure  plus  spanwise  compression  due  to  wing  bending.  The  structural  optimization 
problem  is  formulated  using  the  panel  weight  as  the  objective  function,  with  constraints 
on  stress  and  buckling.  The  spacing  of  the  hat-stilfeners,  the  thickness  of  the  skin,  and  the 
thickness  of  the  components  of  the  hat-stiffener  are  used  as  design  variables.  The  initial 
geometry  of  the  hat-stiffened  panel  design  is  determined  using  the  PANDA2  program,  by 
restricting  the  design  to  have  uniform  cross-section  in  the  spanwise  direction.  Because  of 
the  pressure  loading,  a more  efficient  design  has  variable  cross-section.  Such  designs  are 
obtained  by  combining  the  STAGS  Rankin  el  al.  (1998)  finite  element  program  with  the 
optimization  program  in  the  Microsoft  EXCEL  ( Microsoft  Corporation , 1998) 
spreadsheet  program  using  response  surfaces.  Buckling  and  stress  response  surfaces  are 
constructed  from  multiple  STAGS  analyses  and  are  used  as  constraints  in  the 
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optimization.  The  optimization  conducted  with  the  response  surfaces  results  in 
considerable  weight  savings  compared  to  the  uniform  cross-section  design,  albeit  in  a 
more  complex  design.  Initial  optimization  cycles  identify  a design  space  where  simple 
approximate  analyses,  such  as  Euler-Bernoulli  beam  theory  and  Kirchhoff  plate  theory 
applied  to  laminated  composites,  can  be  used  to  predict  the  behavior  ot  the  structure. 

Introduction 

Major  air  carriers  have  expressed  the  need  for  larger  airplanes  to  meet  the 
growing  demands  for  air  travel,  especially  in  the  Pacific  Rim  and  on  trans-Atlantic  routes 
between  major  airports  in  the  USA  and  in  Europe.  A blended-wing-body  (BWB)  800- 
passenger  aircraft  is  one  of  several  configurations  currently  being  considered  for 
satisfying  this  need.  As  the  name  implies,  a principal  feature  of  a BWB  transport  is  a 
wide  double-deck  centerbody  that  is  blended  into  the  wing.  Due  to  the  shape  of  the 
airplane,  the  pressurized  centerbody  region,  which  includes  both  the  passenger  area  and 
the  cargo  area,  is  non-circular  (see  Figure  16). 


Figure  16:  Center-body  region  of  a blended-wing-body  transport  airplane 
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The  non-circular  centerbody  region  is  challenging  from  the  standpoint  ol 
structural  design,  since  the  upper  and  lower  cover  panels  carry  the  overall  aircraft  body 
moments  Mx,  internal  pressure,  and  wing  bending  moments  My  as  shown  in  Figure  16.  To 
reduce  instability  problems  in  the  upper  cover  panel,  which  is  subjected  to  compressive 
running  loads,  Nx,  due  to  wing  bending,  and  to  reduce  bending  stresses  due  to  the  internal 
pressure  loading,  the  inboard  centerbody  section  is  divided  into  several  passenger  and 
cargo  bays  which  are  separated  by  chordwise  ribs.  A typical  passenger  bay  is  150  inches 
wide,  as  shown  in  Figure  16. 

A variety  of  structural  configurations  for  the  pressurized  upper  cover  panel 
structure  were  considered  as  part  of  a BWB  design  study,  led  by  the  McDonnell  Douglas 
Corporation  (now  The  Boeing  Company,  Long  Beach)  in  collaboration  with  NASA  and 
four  uni  versities  (University  of  Florida,  Stanford  University,  University  of  Southern 
California,  and  Clark  Atlanta  University).  The  configurations  that  were  considered  in  the 
study  include  conventional  skin-stringer  constructions,  deep  sandwich  structure,  and 
separate  structures  for  carrying  the  internal  pressure  and  wing  bending  loads.  A deep 
sandwich  configuration  was  chosen  as  the  baseline  upper  cover  panel  concept,  on  the 
basis  of  low  weight  combined  with  ease  of  fabrication  and  assembly.  However,  some 
unresolved  issues  of  sandwich  structures  might  prevent  their  use  as  primary  structure  for 
a BWB  airplane.  The  present  study  describes  a structural  optimization  study  of  one  of  the 
alternative  structural  concepts  being  considered  for  the  upper  cover  panel  of  a typical 
passenger  bay  of  a BWB  transport  airplane.  Specifically,  results  are  presented  for  a 
composite  hat-stiffened  skin  panel  configuration.  The  skin  and  all  of  the  components  of 
the  hat-stiffener  are  built  up  from  graphite-epoxy,  warp-knit  preforms.  Each  preform  is  a 
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stack  of  material  equivalent  to  seven  layers  of  unidirectional  prepreg  with  0,  45  and  90 
degree  fibers,  and  has  a cured  thickness  of  0.055  inches.  The  loading  cases  used  in  this 
preliminary  study  did  not  include  Mx  shown  in  Figure  16.  The  wing  bending  moment  My 
was  transformed  into  equivalent  Nx  compressive  and  tensile  running  loads  in  the  upper 
and  lower  cover  panels  respectively.  The  weight  of  the  upper  cover  panel  obtained  for  the 
hat-stiffened  panel  concept  was  compared  to  the  thick  sandwich  concept  for  the  load 
cases  available  for  this  study.  A hat-stiffened  panel  design  was  considered  to  be  a valid 
alternative  to  the  sandwich  panel  design  if  the  weight  of  the  hat-stiffened  panel  was  equal 
or  less  than  3.0  Ib/ft2. 

The  structural  optimization  problem  is  formulated  using  the  panel  weight  as  the 
objective  function,  with  stress  and  buckling  constraints.  The  spacing  of  the  hat-stiffeners, 
the  thickness  of  the  skin,  and  the  thickness  of  the  components  of  the  hat-stiffener  are  used 
as  design  variables.  The  thickness  variables  are  integer  multiples  of  the  thickness  of  the 
basic  composite  material  stack  described  above.  Therefore,  the  optimization  problem  is 
discrete.  Furthermore,  the  structural  analyses  required  to  evaluate  the  constraints  are 
performed  by  an  analysis  code  that  does  not  have  optimization  capabilities  and  is  difficult 
to  interface  with  an  optimization  program.  For  such  situations,  response  surface 
techniques,  which  create  simple  approximations  of  structural  responses,  have  been  shown 
to  be  useful  Mason  el  al.  (1994).  Response  surface  techniques  also  permit  simple  analysis 
models  to  be  integrated  with  more  complex  analysis  models.  In  the  present  work,  both 
simple  and  complex  analysis  models  are  integrated  in  response  surfaces  that  are  used  in 
the  structural  optimization. 
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Problem  Description 

The  structural  configuration  considered  in  the  optimization  problem  is  a hat- 
stiffened  skin,  upper  cover  panel  of  a typical  passenger  bay,  as  shown  in  Figure  17. 


Figure  17:  Hat-stiffened  skin  upper  cover  panel  of  the  BWB 

The  panel  is  150  inches  long  in  the  spanwise  (x)  direction,  and  900  inches  long  in 
the  chordwise  (y)  direction.  Two  loading  conditions  are  considered  in  the  design.  The 
first  loading  condition  is  combined  internal  pressure  and  spanwise  (x)  compression.  The 
second  loading  condition  is  internal  pressure-only.  The  two  load  cases  considered  do  not 
have  a load  component  in  the  chordwise  direction  but  they  were  considered  to  be 
sufficient  for  obtaining  a comparison  of  preliminary  design  concepts  for  the  upper  cover 
panel.  The  ends  of  the  stiffened  panel  at  x = 0 inches,  and  at  x = 150  inches  are  clamped. 
The  clamped  boundary  condition  is  achieved  by  two  simple  support  boundary  conditions 
applied  to  the  base  and  to  the  crown  of  the  stiffener.  The  end  of  the  panel  at  x = 0 inches 
is  restrained  from  movement  in  the  x-direction  while  the  opposite  end  of  the  panel  at 
x = 1 50  inches  is  constrained  to  have  a uniform  u displacement.  The  unloaded  edges  of 
the  panel  aty  = 0 inches  and  at>>  = 900  inches  are  simply  supported  with  the  v 
displacement  component  unconstrained. 
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The  skin  and  the  individual  components  of  the  hat-stiffener  are  constructed  from 
graphite-epoxy  warp-knit  preforms,  which  were  developed  under  several  NASA  contracts 
(NASA  Contracts  NASI  18862).  Each  preform  is  a stack  of  material  equivalent  to  seven 
layers  of  unidirectional  prepeg  with  0,  45  and  90  degree  fibers.  The  nominal  stacking 
sequence  of  the  preforms  used  in  the  skin  and  in  all  ol  the  components  of  the  hat-stittener 
is  [45/-45/0/90/0/-45/45].  Each  preform,  or  stack,  has  a cured  thickness  of  0.055  inches. 
Nominal  material  properties  for  a cured  stack  and  the  stress  allowables  that  are  used  in 
the  designs  are  provided  in  Table  1 . 


Table  1 : Material  properties  for  graphite-epoxy  preforms 


En(msi) 

9.25 

E22(msi) 

4.67 

Gi2(msi) 

2.27 

V 12 

0.397 

Sail 

5.40  10'3 

CTall(ksi) 

50 

Taii(ksi) 

18 

p (lb/in3) 

0.0057 

Structural  Optimization 

An  initial  optimum  structural  design  for  the  upper  cover  panel  was  obtained  using 
the  PANDA2  program  Bushnell  (1987).  A refined  optimum  structural  design  was  then 
obtained  by  an  optimization  using  response  surface  techniques.  The  analyses  required  to 
construct  the  response  surfaces  were  performed  using  both  simple  and  complex  analysis 
models.  The  complex  structural  analyses  were  performed  using  the  STAGS  (S  fructural 
Analysis  of  General  Shells)  finite  element  program.  The  initial  design  was  obtained  for  a 
combined  load  case  internal  pressure  p = 14.84  psi,  and  spanwise  compression 
Nx  = -4319  Ib/in.  The  optimization  for  the  initial  load  case  is  described  herein  in  the 
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sections  ’’Design  Using  PANDA2  for  Initial  Load  Condition"  and  "Design  Using 
Response  Surface  Techniques  for  Initial  Load  Condition”.  As  the  airplane  design 
evolved,  the  design  loads  for  the  combined  load  case  were  updated  to  internal  pressure 
p = 15.59 psi  and  spanwise  compression  Nx  = -2879  Ib/in,  and  an  additional  load  case  of 
internal  pressure-only  p = 18.56  psi  was  added.  The  optimum  structural  design  obtained 
for  the  initial  load  case  was  used  as  the  starting  point  in  the  optimization  problem  tor  the 
updated  load  conditions.  The  design  for  the  updated  load  conditions  is  described  herein  in 
the  following  section  “Design  for  Updated  Load  Conditions.”  A flow  chart  of  the  history 
of  the  different  hat-stiffener  cover  panel  structures  obtained  during  the  optimization  is 
shown  in  Figure  1 8. 


Hat  stiffened  composite  panel  optimization  for  a 
blended-wing-body  ( BWB)  airplane 


2nd  Load  Set 

p = 15.59  psi,  Nx  = -2879  Ib/in 
p = 18.56/w/ 

Design  Variables:  tsb,tcb,teb,d,tw 


PANDA2  ' 

► 

Unrounded  Thickness  W = 4.65  Ib/ft2 

Rounded  Thickness  W = 4.80 Ib/ft2 

W 

► 


Response 

Surface 


W = 4.30  Ib/ft2 


D.  Bushnell  W = 3.7 1 Ib/ft2 


-*■  STAGS 


t 


W = 2.75  Ib/ft2 


Figure  18:  Flow  chart  of  the  optimization  history  of  the  hat  stiffened  panel 
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Design  Using  PANDA2  for  Initial  Load  Condition 

The  first  step  in  the  structural  optimization  was  to  use  the  PANDA2  program  to 

obtain  an  initial  design.  PANDA2  was  developed  specifically  to  find  minimum  weight 
designs  of  stiffened,  flat  or  curved  panels,  or  complete  cylindrical  shells  made  of 
laminated  composite  materials.  The  stiffeners  may  run  in  one,  or  in  two  orthogonal 
directions,  and  all  stiffeners  in  one  direction  are  assumed  to  be  identical  and  uniformly 
spaced.  The  panels  can  be  loaded  by  combinations  of  in-plane  loads,  edge  moments, 
normal  pressure  and  temperature.  Constraints  on  the  design  include  global  and  local 
buckling,  stiffener  crippling,  maximum  displacement  due  to  internal  pressure  loading, 
maximum  tensile  or  compressive  stress  along  the  fiber  and  normal  to  the  fibers  in  each 
lamina,  and  maximum  in-plane  shear  stress  in  each  lamina.  PANDA2  calculates  local  and 
general  buckling  load  factors  using  simple  closed  form  expressions  and  inexpensive  finite 
difference  analyses  of  discretized  models  of  panel  cross-sections.  Local  buckling  is 
predicted  from  the  analysis  of  a single  panel  module,  which  is  assumed  to  repeat  several 
times  over  the  panel  width.  A single  panel  module  consists  of  one  stiffener  plus  the  panel 
skin,  with  width  equal  to  the  spacing  between  stiffeners.  For  a typical  stiffener  PANDA2 
finds  the  axial  location  in  which  each  element  of  the  panel  module  has  the  highest 
compressive  stresses,  and  performs  a stability  analysis  based  on  these  values.  The  results 
of  the  stability  analysis  are  then  used  to  size  each  element  of  the  panel  module. 

In  the  present  problem,  the  stiffeners  run  in  the  spanwise  (x)  direction,  and  the 
panel  consists  of  repetitive  modules  in  the  chordwise  (y)  direction  (Figure  17). 
Furthermore,  the  properties  of  the  panel  are  assumed  to  be  uniform  in  the  spanwise  (x) 
direction.  A cross-section  of  a panel  module  and  the  design  variables  used  in  the 
PANDA2  optimization  are  shown  in  Figure  19. 
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Figure  19:  Cross-section  of  a simple  module  and  design  variables  used  for  the  PANDA2 

optimization 


The  objective  function,  the  design  variables  and  the  side  constraints  used  in  the 
PANDA2  optimization  are  listed  in  Table  2. 


Table  2:  PANDA2  optimization  problem  (except  for  weight  dimensions  are  in  inches) 


Objective  function  : min  {weight} 
Design  Variables  : b,  b2,  w,  W2,  h,  ts, 

tf,  tw?  tc 

Side  Constraints: 

6 < b < 24 

(1) 

b2-  0.75b  > 0 

(2) 

4.5  <b2  <_1 8 

(3) 

2<h<6.5 

(4) 

b2  -w2  >2.6 

(5) 

2.9  < w2  <15.4 

(6) 

2.25  < w < 1 1 .75 

(7) 

0.11  <t<  1.1 

(8) 

Buckling  and  stresses 

constraints  in  PANDA2 

- 

(9) 

*See  Figure  19 


The  first  constraint  shown  in  Table  2 ensures  that  there  are  enough  equally  spaced 
stiffeners  in  the  panel  for  a single  module  model  to  give  a good  approximation  to  the 
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local  skin  buckling  mode.  The  second  and  third  constraints  are  recommended  by  the 
PANDA2  user's  guide  to  guarantee  numerical  stability  in  the  solution  procedure.  The 
fourth  constraint  controls  the  height  of  the  hat,  where  the  upper  value  reflects  a 
manufacturing  limit.  The  fifth  and  sixth  constraints  ensure  that  the  flange  is  at  least  1 .3 
inches  wide,  again  for  manufacturing  reasons,  and  the  seventh  constraint  ensures  that  the 
hat-stiffeners  have  reasonable  proportions.  Finally,  the  eighth  constraint  sets  the  upper 
and  lower  thickness  bounds  of  the  panel  skin  and  of  all  elements  of  the  hat-sti  ffener. 
Additional  constraints  on  the  design  included  global  and  local  buckling,  crippling, 
stiffener  pop-off,  maximum  stresses  along  and  normal  to  the  fibers  in  each  lamina,  and 
maximum  in-plane  shear  stresses  within  each  element  of  the  stiffener.  In  evaluating  the 
constraints,  a factor  of  safety  equal  to  1 .3  was  applied  to  the  stresses  and  a factor  of 
safety  equal  to  1.15  was  applied  to  the  buckling  load  factor.  The  buckling  load  safety 
factor  reflects  a design  requirement  that  the  structure  will  not  buckle  between  design  limit 
load  and  design  ultimate  load.  The  optimum  design  for  the  hat-stiffened  upper  cover 
panel  obtained  by  the  PANDA2  program  is  provided  in  the  second  column  in  Table  3. 


Table  3:  PANDA2  original  and  rounded  optimum  design 


Variable 

Original 

Rounded 

Weight  (lb/ft2) 

4.65 

4.86 

b (in) 

13.83 

13.83 

b2  (in) 

8.4 

8.4 

h (in) 

6.5 

6.5 

w (in) 

4.3 

4.3 

w2  (in) 

5.8 

5.8 

ts  (in) 

0.22 

0.22 

tf(in) 

0.34 

0.33 

tw(in) 

0.20 

0.22 

tc(in) 

0.33 

0.33 
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As  shown  in  Table  3,  the  optimum  design  obtained  by  the  PANDA2  program 
weighed  4.65  lb/ft2.  The  optimum  design  obtained  by  PANDA2  features  continuous 
design  variables.  However,  the  thickness  of  the  panel  skin  and  the  thickness  of  the 
individual  components  of  the  hat-stiffener  are  discrete  variables,  since  the  thicknesses  are 
limited  to  integer  multiples  of  the  thickness  of  one  warp-knit  preform.  Therefore,  the 
optimum  design  obtained  by  PANDA2  must  be  rounded  to  the  next  feasible  value  of  the 
discrete  design  variables.  Rounding  off  the  thicknesses  to  the  nearest  discrete  thicknesses 
increased  the  weight  of  the  cover  panel  to  4.86  lb/fr.  The  rounded-o  ff  design  is  shown  in 
the  third  column  of  Table  3.  For  the  rounded-off  design  shown,  the  active  constraints  at 
the  mid-length  of  the  panel  were  fiber  compression  in  the  crown,  web  buckling,  and  axial 
(spanwise  direction)  strain  in  the  crown.  At  the  ends  of  the  panel,  the  active  constraints 
included  local  buckling  of  the  panel  skin  between  the  hat-stiffeners,  local  buckling  ot  the 
panel  skin  under  the  hats,  and  spanwise  compression  in  the  panel  skin. 

Design  Using  Response  Surface  Techniques  for  Initial  Load  Condition 
For  the  loading  and  boundary  conditions  of  the  current  problem,  the  design 

obtained  with  PANDA2  is  conservative.  The  conservative  design  results  from  the 
imposed  requirement  that  the  panel  properties  are  constant  in  the  spanwise  direction, 
while  the  loading  for  the  present  problem  is  variable  in  the  spanwise  direction. 

A more  efficient  design  for  the  stiffened  panel  was  obtained  by  allowing  the 
cross-section  of  the  panel  to  vary  in  the  spanwise  direction.  Based  upon  the  results  of  the 
PANDA2  analysis,  the  panel  was  divided  into  three  sections:  two  identical  sections  at  the 
ends  of  the  panel,  and  a section  in  the  interior  of  the  panel,  as  shown  in  Figure  20. 
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Interior  section  d 


End 
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900  in 
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End 
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Module  of 
A width  b 


Figure  20:  Division  of  the  hat-stiffened  panel  into  three  span  wise  sections 


The  tendency  of  the  skin  to  buckle  near  the  ends  of  the  panel,  which  was  an  active 
constraint  in  the  PANDA2  analysis,  was  reduced  by  adding  a layer  of  material  equal  to 
the  thickness  of  the  flange  to  the  panel  skin  between  the  hat  stiffeners  and  to  the  panel 
skin  under  the  hat  stiffeners.  Furthermore,  the  thickness  of  the  crown  of  the  hat  stiffeners 
in  the  end  sections  of  the  panel  was  allowed  to  be  different  than  the  thickness  of  the 
crown  of  the  hat  stiffeners  in  the  interior  region  of  the  panel.  The  cross-sections  of  a 
panel  module  in  the  end  regions  of  the  panel  and  in  the  interior  region  of  the  panel  are 
shown  in  Figure  21,  respectively. 
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/ “ / 


(a)  End  section 


/ “ / 


(b)  Interior  section 

Figure  21 : Cross-sectional  geometry  of  a variable  thickness  panel 

The  STAGS  {Rankin  et  al.,  1998)  program  was  used  to  obtain  accurate  buckling 
loads  and  stress  predictions  for  the  variable  cross-section  panel  shown  in  Figure  20  and 
Figure  21.  STAGS  is  a finite  element  code  for  the  non-linear  analysis  of  general  shell 
structures  of  arbitrary  shape  and  complexity.  STAGS,  however,  does  not  have  an 
optimization  capability.  Response  surface  techniques,  which  create  simple 
approximations  of  structural  response,  have  been  shown  to  be  useful  for  design 
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optimization  when  the  design  variables  are  discrete  and  when  it  is  difficult  to  integrate 
the  analysis  and  optimization  codes.  In  the  current  study,  STAGS  finite  element  analyses 
were  used  to  construct  response  surface  models  to  approximate  the  panel  buckling 
response  and  to  approximate  the  mid-surface  compressive,  spanwise,  skin  stress  at  the 
thickness  discontinuity  between  the  end  and  the  interior  regions  of  the  panel.  There  was 
very  little  difference  between  the  stresses  on  the  outer  surface  ol  the  skin  and  the  inner 
surface  of  the  skin,  at  the  discontinuity  between  the  end  and  the  interior  regions  of  the 
panel  therefore  the  mid-surface  stresses  were  compared  with  the  maximum  stress 
allowable.  For  the  first  loading  condition  considered  in  this  study,  the  thickness 
discontinuity  was  the  only  region  in  the  hat  stiffened  panel  where  the  stress  constraint 
was  violated.  The  response  surface  approximations  created  were  then  used  as  constraint 
functions  in  an  optimization  procedure  to  minimize  the  panel  weight.  The  optimization 
was  performed  using  a spreadsheet  in  the  Microsoft  EXCEL  ( 1 998)  software  program  that 
allows  discrete  design  variables.  The  optimization  module  in  EXCEL,  called  “SOLVER” 
is  available  in  several  other  spreadsheet  programs.  The  design  variables  used  in  the 
optimization  are  given  in  Table  4.  The  thickness  of  the  flange  (t/  = 0.1 1 in.)  and  the 
thickness  of  the  web  (tw  = 0.22  in.)  were  kept  constant  during  the  optimization  process. 
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Table  4:  Design  variables  used  in  response  surface  approximation  optimization 


Variable 

Variable  meaning 

d 

Distance  from  panel  ends  to 
thickness  discontinuity 

tsb 

Skin  Thickness  near  panel  ends 

tsm 

Skin  Thickness  in  the  interior  of 
the  panel 

tcb 

Crown  thickness  the  near  panel 
ends 

tern 

Crown  thickness  in  the  interior 
of  the  panel 

Both  the  buckling  response  surface  and  the  stress  response  surface  were 
approximated  using  polynomials.  A least  squares  estimate  of  the  unknown  coefficients  in 
the  polynomials  was  obtained  by  evaluating  the  structural  response  (panel  buckling  load 
or  stress  at  the  thickness  discontinuity)  at  a number  of  design  points  that  exceeded  the 
number  of  coefficients  in  the  approximating  polynomials.  The  structural  response  at  the 
individual  design  points  was  determined  using  the  STAGS  finite  element  code.  The 
method  of  analysis  used  to  perform  stress  and  buckling  analysis  was  linear.  In  particular 
the  buckling  analysis  was  conducted  by  performing  a linear  bifurfaction  buckling 
analysis  from  a linear  prestress  state.  The  internal  pressure  p and  the  in-plane  load  Nx 
were  included  in  the  same  load  set  so  that  the  same  buckling  load  factor  was  applied  to 
both  loads.  In  all  of  the  STAGS  analyses,  an  additional  factor  of  safety  equal  to  1.25  was 
applied  to  the  end  load  so  that  a spanwise  compression  load,  Nx  = -5398  lb. /in  was  used. 

In  order  to  keep  the  cost  of  constructing  a response  surface  as  low  as  possible, 
while  ensuring  the  accuracy  of  the  response  surface  model,  care  must  be  taken  in 
selecting  the  design  points.  Several  methods  are  available  for  selecting  the  design  points 
so  that  the  error  in  the  approximation  is  minimized.  In  the  present  study,  the  D- 
Optimality  criterion,  as  implemented  in  the  JMP  program  (SAS  Institute , 1998)  was  used 
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to  select  the  design  points.  The  implementation  in  the  JMP  program  finds  a D-Optimal 
set  of  points  from  a given  set  of  candidate  design  points  in  the  design  domain.  The  9604 
candidate  design  points  that  were  used  for  constructing  the  response  surfaces  are  defined 
in  Table  5. 


Table  5:  Candidate  points  for  response  surface  construction 


Design 

Variable 

Minimum 

(in) 

Step 

(in) 

Maximum 

(in) 

Nominal  Design 
(in) 

d 

12 

12 

48 

36 

tsb 

0.110 

0.055 

0.440 

0.165 

tsm 

0.110 

0.055 

0.440 

0.110 

tcb 

0.110 

0.055 

0.440 

0.220 

tcm 

0.110 

0.055 

0.440 

0.165 

Also  given  in  Table  5 is  a nominal  design,  selected  by  engineering  judgment.  The 
nominal  design  weighted  3. 741  Ib/fr,  had  a buckling  load  factor  applied  to  both  the 
internal  pressure  and  compressive  running  loads  of  1.898,  and  has  spanwise,  compressive 
mid-surface  skin  stress  at  the  thickness  discontinuity  equal  to  20,288  psi. 

Before  proceeding  with  the  selection  of  the  design  points,  the  size  of  the  design 
domain  was  reduced  from  9604  to  740  feasible  design  points  by  introducing  the 
following  constraints  based  on  the  expectations  for  the  optimum  design: 

• The  weight  was  required  to  be  between  3.0  lb/ft  and  4.3  lb/ft  . 

• The  skin  near  the  ends  of  the  panel  was  required  to  be  thicker  than  the  skin  in 
the  interior  of  the  panel  (tsb  > tsm). 

• The  crown  of  the  hat-stiffener  in  the  interior  of  the  panel  was  required  to  be 
thicker  than  the  crown  in  the  end  section  of  the  panel  ( tcm  > tcb). 
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Buckling  Response  Surface  Approximation 

The  STAGS  model  used  in  the  analysis  of  the  hat-stiffened  panel  had 

approximately  70,000  degrees  of  freedom.  The  panel  skin  and  all  of  the  elements  of  the 
stiffeners  were  modeled  as  branched  shells.  One  linear  stress  and  linear  bifurcation 
buckling  analysis  using  this  model  required  approximately  6,500  CPU  seconds  on  a DEC 
ALPHA  200  4/166  work  station. 

Because  of  the  time  required  to  perform  each  of  the  STAGS  structural  analyses,  a 
simple  linear  polynomial  response  surface  approximation  for  the  buckling  load  factor  was 
initially  constructed  first  using  1 1 design  points  (design  points  1-1 1 in  Table  6).  Ten  of 
the  design  points  in  Table  6 (design  points  2-1 1)  were  selected  using  the  D-Optimality 
criterion.  Design  point  1,  the  nominal  design  given  in  Table  5,  was  selected  by 
engineering  judgement. 
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Table  6:  Structural  designs  used  for  the  linear  response  surface 


Point  # 

d 

kb 

tsm 

km 

kb 

weight 

X, 

G max 

(in) 

(in) 

(in) 

(in) 

(in) 

(lb/ft.) 

(psi) 

1 

36 

0.165 

0.110 

0.220 

0.165 

3.741 

1.898 

20288 

2 

12 

0.110 

0.110 

0.110 

0.110 

3.110 

0.554 

43031 

3 

12 

0.110 

0.110 

0.440 

0.110 

3.747 

0.531 

42969 

4 

12 

0.220 

0.220 

0.220 

0.220 

4.237 

2.262 

23780 

5 

12 

0.440 

0.110 

0.440 

0.110 

4.166 

0.821 

25978 

6 

12 

0.440 

0.110 

0.440 

0.440 

4.288 

0.987 

27846 

7 

12 

0.440 

0.220 

0.110 

0.110 

4.264 

0.991 

23330 

8 

48 

0.110 

0.110 

0.440 

0.440 

4.206 

1.232 

- 

9 

48 

0.165 

0.165 

0.440 

0.110 

4.158 

1.207 

14295 

10 

48 

0.275 

0.110 

0.110 

0.110 

4.288 

1.010 

15164 

11 

12 

0.165 

0.165 

0.220 

0.165 

3.965 

1.296 

- 

12 

48 

0.220 

0.055 

0.275 

0.220 

4.153 

0.727 

27328 

13 

48 

0.220 

0.055 

0.275 

0.110 

3.991 

0.749 

24871 

14 

48 

0.165 

0.110 

0.165 

0.220 

3.314 

2.020 

17250 

15 

48 

0.110 

0.165 

0.165 

0.220 

3.816 

1.058 

12850 

16 

48 

0.220 

0.165 

0.165 

0.110 

4.215 

1.460 

17514 

17 

24 

0.220 

0.165 

0.275 

0.220 

4.142 

2.492 

21976 

18 

24 

0.220 

0.165 

0.165 

0.110 

3.890 

2.070 

21408 

19 

24 

0.220 

0.055 

0.165 

0.110 

3.294 

0.201 

39861 

20 

24 

0.110 

0.165 

0.275 

0.110 

3.781 

1.084 

20679 

21 

24 

0.110 

0.055 

0.275 

0.110 

3.185 

0.230 

39997 

22 

24 

0.110 

0.055 

0.165 

0.220 

3.095 

0.205 

44642 

23 

24 

0.110 

0.055 

0.165 

0.110 

3.014 

0.196 

44023 

24 

48 

0.110 

0.055 

0.165 

0.110 

3.339 

1.131 

27607 

25 

24 

0.165 

0.055 

0.165 

0.110 

3.154 

0.199 

39004 

26 

48 

0.220 

0.165 

0.220 

0.110 

4.261 

1.328 

12139 

27 

48 

0.110 

0.110 

0.165 

0.110 

3.497 

1.079 

18363 

28 

24 

0.220 

0.110 

0.165 

0.110 

3.592 

1.058 

23939 

29 

24 

0.165 

0.055 

0.165 

0.110 

3.154 

0.199 

- 

30 

48 

0.220 

0.165 

0.220 

0.110 

4.261 

1.328 

- 

31 

48 

0.110 

0.110 

0.165 

0.11 

3.497 

1.079 

- 

32 

24 

0.220 

0.110 

0.165 

0.11 

3.592 

1.058 

- 

33 

36 

0.165 

0.110 

0.165 

0.11 

3.612 

1.065 

- 

The  linear  fit,  constructed  using  these  1 1 points  spanning  through  the  entire 
design  domain,  for  predicting  the  buckling  load  factor  was  very  poor  with  an  estimate  of 

'y  • • 

the  root  mean  square  error  RMSE  = 0.61 , and  R Adj  = 0.046.  Since  the  buckling  load 
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factor  is  approximately  equal  to  1 .0,  this  RMSE  error  is  very  large.  The  parameter  R Adj 
indicates  the  quality  of  a fit.  A perfect  fit  is  indicated  by  R Adj  = 7.0,  and  R Adj  = 0.0 
represents  a very  poor  fit. 

In  an  attempt  to  improve  the  accuracy  of  the  linear  response  surface 
approximation,  a new  design  domain  was  defined  around  the  nominal  design.  The 
thickness  variables  were  permitted  to  change  by  +0.055  inches  from  the  nominal  design 
and  the  length  of  the  end  section  d , was  permitted  to  change  by  +12  inches.  Thus,  each 
variable  could  have  3 possible  values  and  a total  of  243  design  points  was  generated  (81 
of  the  generated  points  violated  the  tsm  thickness  constraint  since  the  nominal  design  was 
at  the  lower  bound  for  this  design  variable).  Twelve  design  points  were  selected  out  of 
these  243  points  using  the  D-Optimality  criterion  (points  12-23  in  Table  6). 

The  linear  response  surface  for  the  buckling  load  factor  obtained  using  the  12  new 
points  had  an  RMSE  = 0.54  and  R Adj  = 0.56,  indicating  that  the  fit  was  still 
unsatisfactory.  Moreover,  the  tslal  as  defined  in  Eq.  (3.42)  of  the  coefficients  were  very 
small.  The  ts,a,  is  a parameter  that  indicates  the  confidence  in  the  values  of  the 
coefficients  obtained  and  it  is  defined  as  the  ratio  between  the  value  of  a coefficient  and 
its  variance.  A higher  confidence  is  indicated  by  a higher  value  of  the  tslah  which  means 
that  the  value  of  the  coefficient  is  much  larger  than  its  variance.  Since  the  linear  response 
surface  models  were  not  satisfactory,  1 0 design  points  were  added  to  the  23  design  points 
that  were  used  in  constructing  the  two  linear  response  surface  models  described  above. 
Using  all  the  33  design  points  in  Table  6,  a full  quadratic  polynomial  was  fitted  over  the 
entire  design  domain.  A full  quadratic  polynomial  in  five  design  variables  has  21 
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coefficients.  Terms  in  the  polynomial  with  a low  tsta,  were  discarded  as  long  as  R~ Adj  kept 
increasing.  The  quadratic  polynomial  retained  ten  coefficients,  and  is  given  by: 

^ = -4.199+  0.1708  d + 34.14  tsm - 0.001362  d 2 

- 31.98  tj - 0.4712  d tsm+  72.30  tsm  txb-  95.94  tj  (4.1) 

- 0-  759  tcm  d + 22.43  tcm  tsh 

For  the  polynomial  fit  of  Eq.  (4.1),  RMSE  = 0.25  and  R2 Adj  = 0.82.  Furthermore, 
the  lowest  tstat  was  3. 1 7,  which  indicates  reasonable  confidence  in  the  coefficients.  The 
accuracy  of  the  response  surface  approximation  was  also  checked  by  constructing  several 
response  surfaces  using  only  32  points,  and  then  comparing  the  response  surface 
predictions  at  the  point  left  out  with  the  STAGS  analysis  predictions  at  that  point  (a 
procedure  known  as  PRESS,  Eq.  (3.33)).  Based  upon  these  comparisons,  the  quadratic 
response  surface  approximation  for  the  buckling  load  factor  was  expected  to  have  less 
than  20%  error. 

Stress  Response  Surface. 

A quadratic  response  surface  approximation  for  the  mid-surface  compressive, 
spanwise  stress  in  the  skin,  at  the  change  in  skin  thickness  between  the  end  sections  of 
the  panel  and  the  interior  section  of  the  panel,  was  also  constructed.  Stress  values  at  all 
other  locations  in  the  panel,  for  all  of  the  designs  analyzed,  did  not  exceed  the  maximum 
stress  allowable.  The  response  surface  approximation  obtained  with  units  of  (ksi)  is  given 


by: 
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cr=  1.087-  2.011  d - 95.15  tsb  - 48L5  txm 

+ 0.01503 d2  - 1.141  d tsb+  2.816  tsmd  (4.2) 

+ 323.4  tsm  tsb  + 813.6  t)m 

with,  R2Adj  = 0.96  and  RMSE  = 1924  psi.  The  tstat  for  all  of  the  coefficients  were 
higher  than  3.33.  The  accuracy  of  the  response  surface  approximation  was  also  checked 
by  the  PRESS  procedure,  which  indicated  that  a maximum  error  of  20%  is  expected  when 
the  stress  response  surface  is  used  to  estimate  the  mid-surface  compressive,  spanwise 
stresses  in  the  skin  near  the  thickness  discontinuity. 

Optimization  Using  the  Response  Surfaces  Approximations 

As  indicated  above,  the  response  surface  approximation  for  the  buckling  load 

factor,  X,  and  the  stress  response  surface  had  errors  of  20%.  Therefore  the  required 
buckling  load  factor  was  increased  by  20%  from  1.15  to  1 .38,  and  the  safety  factor  on  the 
compressive  stresses  was  increased  by  20%  from  1 .3  to  1 .56. 

The  optimization  problem  was  formulated  as  shown  in  Table  7 and  solved  using  a 
reduced  gradient  optimizer  available  in  Microsoft  EXCEL. 


100 


Table  7:  Initial  optimization  problem  using  response  surfaces 


Objective  Function: 

Min{  weight} 

Design  Variables: 

d,  tsb,  tsrn,  tcm,  tcb 

Constraints: 

X.  > 1 .38 

a <32051  (psi) 

tsb  — Cm  ^ 0 

tcm  — tcb  ^ 0 

24  < d < 48  (in.) 

0.1 10  < tsb  <0.220  (in) 
0.1 10  < tsm  <0.165  (in) 
0.165  < tcm  < 0.275  (in) 
0.1 10  < tcb  < 0.220  (in) 

The  optimum  design  obtained,  subject  to  the  constraints  specified  in  Table  7,  is 
presented  in  Table  8.  Analysis  ol  the  design  in  1 able  8 with  S TAGS  gave  a buckling  load 
factor,  X - 1.379,  and  a maximum  spanwise,  stress,  cr=  -22,450 psi,  in  the  skin  at  the 
boundary  between  the  end  section  and  the  interior  section.  The  variable  cross-section 
design  for  the  upper  cover  panel,  obtained  using  the  response  surface  approach  has  a 
weight  of  3.53  lb/ft2  (Table  8)  compared  to  a weight  of  4.86  Ib/fr  (Table  3)  for  the 
uniform  cross  section  panel  design. 


Table  8:  Initial  optimum  design  obtained  using  response  surface 


Parameter 

Optimum 

d (in) 

30 

tsb  (in) 

0.165 

tsm  (in) 

0.110 

tcm  (in) 

0.165 

tcb  (in) 

0.110 

Weight  (lb/ft2) 

3.533 

X STAGS 

1.379 

a (psi.)  STAGS 

22,450 

X response  surface 

1.416 

a (psi.)  response  surface 

23,845 
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Design  for  Updated  Load  Conditions 

After  the  optimization  described  above  was  completed,  the  design  loads  were 
updated  as  the  overall  airplane  design  changed.  A new  load  case  of  internal  pressure-only 
p = 18.56  psi  was  added,  and,  in  the  combined  load  case,  the  pressure  loading  was 
changed  from  14.84 psi  to  15.59 psi  and  the  in-plane,  spanwise  load  was  changed  from 
4319  lb/ in  to  2879  Ib/in.  In  addition,  the  buckling  and  stress  safety  factors  were  reduced 
to  1 .0.  The  reduction  in  the  buckling  safety  factor  reflected  a change  in  design 
philosophy,  allowing  local  buckling  of  the  skin  between  the  design  limit  load  and  the 
design  ultimate  load. 

PANDA2  was  used  to  optimize  the  hat  stiffened  panel  with  uniform  cross-section 
for  the  updated  combined  load  case,  and  a panel  weight  of  4.30  lb/fr  was  obtained, 
indicating  that  the  updated  loading  condition  was  less  severe  than  the  initial  loading 
condition.  Moreover,  Dr  David  Bushnell,  the  developer  of  PANDA2,  obtained  an  optimal 
two  cross-section  hat  stiffener  panel  using  PANDA2.  For  a transition  specified  at 
distance  of  24.0  inches,  the  weight  obtained  by  PANDA2  was  3.71  lb/ft2 . 

To  obtain  a variable  cross-section  design,  STAGS  was  again  used.  The  STAGS 
finite  element  model  of  the  hat  stiffened  panel  was  simplified  to  include  only  one  panel 
module,  13.83  inches  wide,  with  symmetry  conditions  imposed  along  the  sides  parallel  to 
the  x-axis.  Also,  the  weight  calculations  were  refined  to  remove  duplications  in  the 
previous  calculations  due  to  intersecting  finite  elements. 

The  design  for  the  updated  load  conditions  was  initiated  using  the  optimum 
design  obtained  in  the  design  cycle  for  the  initial  load  condition  as  a starting  point.  In  the 
optimization  for  the  initial  load  condition,  the  thickness  of  the  skin  in  the  interior  section 
of  the  panel,  tsm,  remained  at  its  lower  limit  throughout  the  optimization  process  and  was 
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not  an  active  variable  in  the  design.  In  addition,  for  the  optimum  design  presented  in 
Table  8,  the  webs  account  for  approximately  40%  of  the  total  panel  weight.  Based  upon 
these  observations,  the  design  variable  list  was  modified  by  including  the  web  thickness, 
tw,  as  a design  variable  and  by  removing  the  thickness  of  the  skin  in  the  interior  section  of 
the  panel,  tsm,  from  the  design  variable  list. 

A design  domain  for  the  updated  optimization  was  generated  around  the  optimum 
design  point  obtained  in  the  previous  optimization.  The  design  domain  was  limited  to  162 
new  design  points  by  introducing  the  following  restrictions.  The  thickness  of  the  skin  in 
the  end  sections  of  the  panel  and  the  thickness  of  the  crown  in  the  interior  section  of  the 
panel,  were  permitted  to  change  by  +0.055  inches  from  the  previous  optimum  design 
(Table  8).  The  thickness  of  the  crown  in  the  end  sections  of  the  panel  was  allowed  to  be 
twice  as  thick  as  it  was  for  the  initial  load  case  in  order  to  handle  the  new  load  case  of 
internal  pressure-only  that  induces  high  tensile  stresses  in  the  crown  near  the  panel  ends. 
The  web  was  constrained  to  have  a thickness  equal  to  0. 1 65  inches  or  0.220  inches.  A 
thinner  web  would  violate  the  buckling  constraint,  and  a thicker  web  would  result  in 
unacceptably  heavy  designs.  The  distance  from  the  panel  ends  to  the  thickness 
discontinuity,  d , was  permitted  to  change  by  + 6 inches  from  the  previous  optimum 
design  value.  The  thickness  of  the  skin  in  the  interior  section  of  the  panel,  tsm,  was  equal 
to  0.1 10  inches;  and  tsm  was  held  constant  during  the  optimization  process.  The  flange 
thickness,  which  was  equal  to  0.1 10  inches,  was  also  held  constant  during  the 
optimization  process. 

The  JMP  program  was  used  to  select  a D-Optimal  set  of  26  new  design  points 
from  the  162  candidate  design  points.  STAGS  was  used  to  conduct  linear  analyses  of 
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each  of  the  26  designs  for  the  internal-pressure-only  load  case  and  for  the  combined  load 
case.  Table  9 shows  the  26  designs  and  the  results  obtained  from  the  STAGS  analyses  for 
the  buckling  load  factors  A.p  and  Xc  for  the  internal  pressure-only  load  case  and  for  the 
combined  load  case,  respectively. 


Table  9:  Structural  designs  near  the  initial  optimum  design  that  was  used  for  the  updated 


response  surface  optimization 


Point  # 

d 

(in) 

tsb 

(in) 

1cm 

(in) 

tcb 

(in) 

t\V 

(in) 

Weight 

(lb/ft2) 

A,p 

Xq 

^Max 

(psi) 

1 

30 

0.220 

0.110 

0.110 

0.220 

3.505 

0.848 

0.805 

56,858 

2 

30 

0.165 

0.165 

0.165 

0.220 

3.410 

1.680 

1.496 

46,161 

3 

30 

0.110 

0.110 

0.110 

0.165 

2.755 

0.661 

0.620 

- 

4 

36 

0.220 

0.220 

0.165 

0.165 

3.366 

2.565 

1.756 

- 

5 

36 

0.220 

0.110 

0.165 

0.165 

3.321 

0.800 

0.695 

- 

6 

36 

0.110 

0.220 

0.165 

0.220 

3.341 

1.000 

0.893 

- 

7 

36 

0.110 

0.220 

0.165 

0.165 

2.943 

0.895 

0.779 

- 

8 

36 

0.110 

0.110 

0.165 

0.220 

3.186 

0.865 

0.797 

- 

9 

36 

0.110 

0.110 

0.165 

0.165 

2.786 

0.761 

0.652 

- 

10 

24 

0.220 

0.220 

0.165 

0.220 

3.602 

2.736 

1.206 

- 

11 

24 

0.220 

0.220 

0.165 

0.165 

3.207 

2.516 

1.003 

- 

12 

24 

0.220 

0.110 

0.165 

0.220 

3.399 

0.870 

0.803 

- 

13 

24 

0.220 

0.110 

0.165 

0.165 

3.002 

0.716 

0.657 

- 

14 

24 

0.110 

0.220 

0.165 

0.220 

3.322 

1.005 

0.906 

- 

15 

24 

0.110 

0.110 

0.165 

0.220 

3.120 

0.832 

0.764 

- 

16 

36 

0.220 

0.165 

0.110 

0.165 

3.290 

1.452 

1.238 

64,761 

17 

36 

0.220 

0.110 

0.110 

0.165 

3.212 

0.694 

0.664 

66,607 

18 

36 

0.165 

0.220 

0.110 

0.220 

3.553 

1.698 

1.508 

55,149 

19 

36 

0.110 

0.110 

0.110 

0.220 

3.189 

0.814 

0.769 

58,858 

20 

30 

0.165 

0.165 

0.110 

0.220 

3.420 

1.661 

1.484 

57,009 

21 

30 

0.110 

0.220 

0.110 

0.165 

3.334 

0.892 

0.890 

63,389 

22 

30 

0.220 

0.220 

0.110 

0.220 

4.000 

2.705 

1.260 

54,627 

23 

24 

0.165 

0.110 

0.110 

0.165 

3.262 

0.665 

0.623 

66,618 

24 

24 

0.110 

0.220 

0.110 

0.165 

3.324 

0.893 

0.785 

61,609 

25 

24 

0.110 

0.220 

0.110 

0.165 

2.926 

0.971 

0.874 

56,235 

26 

24 

0.110 

0.110 

0.110 

0.165 

2.722 

0.650 

0.607 

67,113 

The  buckling  load  factors  were  obtained  from  a linear  bifurcation  buckling 
analysis  from  a linear  prestress  state.  For  the  combined  load  case,  the  internal  pressure 
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load  and  spanwise  compressive  load  were  included  on  one  load  set  and  the  buckling  load 
was  applied  to  the  combined  load.  The  pressure-only  load  generated  high  tensile  stresses 
in  the  crown  of  the  hat  stiffener  due  to  bending.  The  tensile  mid-surface  spanwise  stresses 
in  the  crown  for  the  internal  pressure-only  load  case,  calculated  at  3.5  inches  from  the 
panel  end,  are  also  provided  in  Table  9.  The  dashes  in  Table  9 indicate  designs  for  which 
stress  calculations  were  not  performed. 

The  stress  calculations  performed  with  STAGS  indicated  that  the  stress  allowable 
was  exceeded  in  the  crown  of  the  hat  stiffener,  for  the  internal  pressure-only  load  case 
only.  For  this  load  case,  several  designs  failed  due  to  large,  spanwise,  tensile  stresses  in 
the  crown  of  the  hat  stiffener  near  the  end  sections  of  the  panel.  Furthermore,  the  STAGS 
analyses  showed  that  for  the  two  load  cases  considered  in  the  present  analysis,  a simple 
beam  analysis  provides  a good  approximation  of  the  stress  distribution  along  the  panel 
length.  Since  the  stress  calculations  performed  using  the  beam  approximation  are 
inexpensive,  the  approximate  beam  analysis  was  used  to  calculate  the  stresses  in  the 
crown  of  the  hat  stiffener  at  3.5  inches  from  the  panel  ends,  for  all  of  the  162  structures. 
The  stress  results  from  the  approximate  beam  analysis  were  compared  with  the  STAGS 
analysis  stress  results  for  13  structures  (points  1-2  and  points  16-26  in  Table  9.  The 
difference  between  the  simple  beam  predictions  and  the  STAGS  analysis  predictions 
ranged  from  0.1%  to  6.5%. 

Approximate  Buckling  Analysis. 

The  STAGS  analyses  for  the  26  hat-stiffened  cover  panel  design  provided  in 
Table  9 showed  that  for  buckling  load  factors  less  than  1 .0  or  close  to  1 .0,  buckling  was 
generally  local  and  confined  to  a single  element  of  the  hat  stiffened  panel  (i.e.,  only  the 
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cap  of  the  hat  stiffener  or  the  panel  skin  had  any  appreciable  deformation).  This 
observation  is  demonstrated  in  Figure  22,  where  buckling  mode  shapes  are  shown  for 
four  design  points. 


(a)  Design  Point  3,  combined  internal  pressure  and  in-plane  load  case 


(b)  Design  point  5,  internal  pressure  load  case 


(c)  Design  point  11,  combined  internal  pressure  and  in-plane  compression 

load  case 


(d)  Design  point  2,  combined  load  case 

Figure  22:  Examples  of  buckling  modes  predicted  by  STAGS  analyses.  Design  point 

numbers  refer  to  Table  9 


The  first  buckling  mode  shape  for  the  configuration  defined  by  design  point  3 and 
subjected  to  combined  internal  pressure  and  spanwise  compression  loading  is  shown  in 
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Figure  22a.  The  buckling  is  mostly  confined  to  the  cap  of  the  hat  stiffener  in  the  interior 
section  of  the  panel.  The  first  buckling  mode  for  the  configuration  defined  by  design 
point  5,  and  subjected  to  internal  pressure  only  is  shown  in  Figure  22b.  For  this  design 
and  load  case  buckling  is  confined  mostly  to  the  skin  near  the  panel  ends.  Figure  22c 
presents  an  example  of  skin  buckling  in  the  thin  section  of  the  skin  near  the  skin 
thickness  discontinuity.  Interactive  buckling  modes  were  also  predicted,  but  the  buckling 
load  factors  for  these  modes  were  substantially  greater  than  1 .0.  An  example  of  an 
interactive  buckling  mode  shape  is  shown  in  Figure  22d,  where  both  the  cap  and  the  webs 
of  the  hat  stiffener  buckle  simultaneously. 

Since  individual  components  of  the  hat-stiffened  panel  buckle  independently  for 
buckling  load  factors  close  to  1 .0,  an  approximate  buckling  analysis  was  developed  by 
representing  a subregion  of  each  component  of  the  hat  stiffened  panel  as  a simply 
supported  plate.  These  subregions  were  assumed  to  have  width,  w,  equal  to  the  width  of 
the  component  of  the  stiffener  they  were  representing  (e.g.,  the  width  w = 4.3  in.  was 
used  for  the  crown  of  the  hat  stiffener),  and  length  equal  to  the  length  of  a half  wave  of 
the  bucking  mode  that  corresponds  to  the  lowest  critical  load  value  of  a plate  of  width  w 
that  is  simply  supported  on  all  edges.  The  component  with  the  lowest  buckling  load 
determined  the  buckling  load  for  the  hat-stiffened  panel. 

The  spanwise  stresses  in  the  panel  induced  by  the  internal  pressure  load  vary 
quadratically  along  the  length  of  the  panel.  However,  the  stress  is  approximately  constant 
over  a single  buckling  half  wave,  since  the  length  of  a single  buckling  half  wave  is  much 
smaller  than  the  length  of  the  panel,  (see  Figure  22).  Therefore,  the  buckling  stress  for  a 
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single  component  of  the  hat  stiffened  panel  was  approximated  from  the  buckling  solution 


for  a simply  supported  plate  under  constant  load,  that  is  given  by: 


n 


G 


budi  2i  2 d2 

t m bh  K 


[m4Du+2  (Dl2+  2 £>66 \nmK)2  + D22(nR)A ] 


(4.3) 


where  bb  and  t are  the  width  and  the  thickness  respectively  of  the  component  of  the  hat 


stiffened  panel  under  consideration,  R = — is  the  ratio  between  the  length  and  the  width 


of  the  panel  component,  and  m and  n are  respectively  the  number  of  half  waves  in  the 
direction  parallel  and  perpendicular  to  the  loading  Nx.  It  can  be  shown  that  the  minimum 
buckling  stress  for  a simply  supported  plate  with  fixed  width  bb  and  thickness  t is 
achieved  for  a length  cib'. 


ab  = 


/ \l/4 

' n ' 

V A22  ) 


(4.4) 


Substituting  Eq  (4.4),  and  the  material  properties  given  in  Table  1 into  Eq.  (4.3) 
gives  the  approximate  buckling  stress  for  an  individual  component  of  the  hat  stiffened 
panel: 


<rb«ck=  2-25  10 


8 


/ V 


\bhj 


(4.5) 


An  approximation  for  the  buckling  load  factor  was  obtained  by  dividing  the 
approximate  buckling  stress  given  by  Eq.  (4.5)  by  the  approximate  applied  stress, 
determined  from  the  simple  beam  analysis  discussed  above.  The  approximate  buckling 


cr 


load  factor  is  defined  as  A = . A value  of  bh  of  8.0  inches  was  used  in  the 


a 


applied 


calculations  of  ahuck  for  the  skin  near  the  ends  of  the  panel  and  for  the  skin  at  the 


thickness  discontinuity  (b  - w in  Figure  21).  To  evaluate  the  buckling  stress  in  the  crown 
of  the  hat  stiffener,  the  value  of  bh  was  set  equal  to  4.3  inches  (w  in  Figure  21). 
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Comparison  of  the  buckling  load  factor  obtained  using  the  approximate  analysis 
with  the  buckling  load  factor  predicted  by  STAGS  for  the  26  designs  provided  in  Table  9 
indicated  that  the  approximate  buckling  load  factor  underestimated  the  panel  buckling 
resistance.  This  result  was  expected  since  the  neighboring  plate  elements  provide  more 
boundary  restraint  than  the  one  provided  by  simple  support  boundary  condition.  Also  the 
stresses  are  not  constant  over  the  length  of  a component  subregion,  and  the  buckling  load 
factor  was  calculated  in  the  approximate  analysis  on  the  basis  of  the  maximum  stress  over 
the  length  ab.  The  buckling  load  factors  predicted  by  the  approximate  analysis  were  up  to 
75%  smaller  than  the  buckling  load  factors  predicted  by  STAGS.  To  improve  the 
buckling  load  factor  predictions  obtained  using  the  approximate  beam-plate  analysis,  the 
STAGS  results  were  used  to  fit  a scale  factor  (as  a single-term  response  surface 
approximation)  to  the  buckling  load  factor  in  each  critical  region  of  the  panel.  The 
computed  scale  factors  are  provided  in  Table  10. 


Table  10:  Scale  factors  for  combined  load  case  buckling  load  factor  Xc 


Buckling  location 

Scale  factor 

Cap  in  the  interior  section 

1.804 

Skin  in  the  interior  section 

3.943 

Skin  at  the  end  section 

1.313 

The  combined  load  condition  was  critical  for  buckling  for  all  of  the  designs 
considered,  so  the  response  surface  was  generated  for  this  load  case  only.  The  buckling 
load  factors  predicted  by  the  approximate  analysis  combined  with  the  scale  factors 
provided  in  Table  10,  had  a maximum  error  of  10%  for  panels  whose  buckling 
deformations  were  primarily  localized  to  one  component  of  the  hat-stiffened  panel.  The 
approximate  analysis  generally  overestimated  the  buckling  load  factor  for  panels  that 
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activate  an  interactive  buckling  response.  Fortunately  these  structures  had  buckling  load 
factors  that  were  well  above  1 .0. 

Minimum  Weight  Structure. 

The  four  three-level  design  variables,  (d,  tsb,  tcb,  tcm)  and  two-level  design  variable 
tw,  generate  a design  domain  of  162  points.  All  162  designs  were  analyzed  using  beam 
theory  to  determine  the  stresses,  and  the  scaled  plate-beam  approximation  to  determine 
the  buckling  load  factor.  The  most  promising  design  was  identified  and  checked  by 
STAGS  analysis.  The  optimum  design,  found  by  inspection  of  the  approximate  analysis 
predictions,  is  given  in  Table  11. 


Table  1 1 : Minimum  weight  structural  design 


Design 

Variable 

Value 

d (in) 

24 

tsb  (in) 

0.165 

tcm  (ifr) 

0.165 

tcb  (in) 

0.220 

tw  (in) 

0.165 

Weight  (lb/ft2) 

2.755 

The  structure  shown  in  Table  1 1 weighs  2. 75  lb/ft2 . The  maximum  spanwise  stress 
predicted  by  the  approximate  analysis  was  a tensile  stress  of  44,101  psi,  in  the  crown  of 
the  hat  stiffener,  for  the  pressure-only  load  condition.  The  minimum  approximate 
buckling  load  factor  was  for  the  combined  load  condition  X = 0.97.  The  buckling  load 
corresponds  to  skin  buckling  in  the  interior  region  of  the  panel.  The  design  provided  in 
Table  1 1 was  further  checked  for  adequacy  using  a STAGS  linear  bifurcation  buckling 
analysis  from  both  a linear  prestress  state  and  a nonlinear  prestress  state.  Both  analyses 
were  concluded  for  the  combined  load  case,  with  both  the  internal  pressure  load  and  the 
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in-plane  compression  load  applied  in  one  load  set.  The  buckling  load  factor  obtained 
from  the  analysis  is  therefore  applied  to  both  loads.  The  STAGS  linear  bifurcation 
buckling  analysis  from  a linear  prestress  state  predicted  a buckling  load  factor,  X = 0.93. 
The  buckling  mode  was  predominantly  buckling  of  the  skin  at  the  axial  location  where 
the  cross-section  changes  and  is  consistent  with  the  buckling  mode  predicted  by  the 
approximate  analysis.  The  predicted  maximum  tensile  stress  in  the  crown  of  the  hat 
stiffener  near  the  ends  was  45,097  psi. 

A linear  bifurcation  analysis  executed  from  a nonlinear  prestress  state,  and  with 
both  loads  applied  on  one  load  set,  predicted  a buckling  load  factor  Xc=  1.11.  The 
corresponding  buckling  mode  shape  is  shown  in  Figure  23. 


z 

Figure  23:  Linear  bifurcation  buckling  mode  from  a nonlinear  prestress  state  of  optimal 

design  for  combined  loading  condition 

The  buckling  mode  is  interactive,  and  the  crown  and  the  web  of  the  hat  stiffener 
buckled  simultaneously.  The  interactive  buckling  mode  predicted  by  the  bifurcation 
analysis  conducted  from  a nonlinear  prestress  state  Xc  corresponds  to  the  ninth  buckling 
mode  found  by  the  bifurcation  analysis  conducted  from  a linear  prestress  state  and  its 
value  was  Xc  = 1 .21 . The  buckling  load  factor  predicted  by  the  bifurcation  analysis  from  a 
nonlinear  prestress  state  is  14%  higher  then  the  buckling  load  factor  predicted  from  a 
bifurcation  buckling  analysis  executed  from  a linear  prestress  state. 
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Concluding  Remarks  for  Structural  Optimization  of  the  Hat  Stiffener  Panel 

A design  study  for  structural  optimization  of  a hat-stiffened  laminated  composite 

panel  concept  for  the  upper  cover  panel  of  a typical  passenger  bay  of  a blended- 
wing-body  transport  plane  is  presented.  The  hat-stiffened  panel  was  subjected  to  internal 
pressure  load,  and  to  combined  internal  pressure  and  in-plane  loads.  The  structural 
optimization  problem  w7as  formulated  using  weight  as  the  objective  function  with  stress 
and  buckling  constraints.  The  initial  optimization  was  conducted  for  a design  having 
uniform  properties  in  the  spanwise  direction  using  the  PANDA2  program.  The  design 
was  then  refined  using  a more  flexible  design  procedure  that  combined  the  STAGS  finite 
element  analysis  program  with  response  surface  approximations  for  the  stresses  and  for 
the  buckling  loads. 

The  design  loads  and  safety  factors  were  changed  during  the  design  process,  but 
the  knowledge  of  the  design  space  gained  from  the  first  load  case  allowed  an  easy  update 
to  the  optimum  design.  Furthermore,  in  the  small  region  in  the  design  space  that  was  used 
for  updating  the  design,  simple  beam  and  plate  approximations  were  used  with  simple 
correction  factors  obtained  from  STAGS  to  generate  response  surface  approximations  for 
the  stresses  and  for  the  buckling  loads. 

The  design  study  demonstrated  the  use  of  three  levels  of  analysis  models  and  the 
use  of  response  surface  approximations  for  integrating  the  analysis  models  into  the  design 
process.  This  process  resulted  in  a reduction  in  the  weight  of  the  hat-stiffened  laminated 
composite  panel  concept  from  4.30  Ib/ft2  to  2. 75  Ib/ft2 . The  hat  stiffener  panel  concept 
proved  to  be  a valid  alternative  to  the  thick  sandwich  concept  since  it  achieved  a weight 
of  less  than  3. 0 Ib/ft2 . 


CHAPTER  5 

MULTIFIDELITY  DESIGN  OF  A COMPOSITE  PANEL  WITH  A CRACK 


Summary 

Crack  propagation  is  an  important  concern  in  the  design  of  aircraft  composite 
fuselage  and  wing  panels.  However,  computer  simulation  of  crack  propagation  is 
computationally  expensive.  This  chapter  proposes  combining  high-fidelity  models  of 
analysis  and  low-fidelity  models  to  calculate  the  crack  propagation  constraint  in  the 
design  optimization  process.  The  strategy  uses  correction  response  surfaces  to  relate  the 
high-fidelity  models  to  the  low-fidelity  models.  Four  different  forms  of  correction 
response  surface  approximation  are  explored  and  their  prediction  capabilities  compared. 
The  multifidelity  approach  is  found  to  be  more  accurate  than  single  fidelity  response 
surface  method  for  the  same  computational  cost. 

Introduction 

The  next  generation  of  supersonic  transport  planes  are  expected  to  fly  at  very  high 
altitudes  and  are  very  vulnerable  to  pressure  loss  due  to  large  cracks,  such  as  due  to 
turbine  blade  penetrations.  Consequently,  accurate  prediction  of  crack  propagation  is  a 
major  issue  in  the  structural  design  of  such  aircraft.  Accurate  calculation  of  crack 
propagation  requires  detailed  structural  models  for  computing  stress  intensity  factors, 
which  may  be  too  expensive  computationally  for  structural  design  optimization. 

The  problem  of  optimization  with  high  cost  models  is  common  to  many 
engineering  design  problems.  Recently,  there  has  been  growing  interest  in  taking 
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advantage  of  the  simple  models  that  were  in  use  a generation  ago  when  computers  were 
less  powerful.  These  models,  which  are  less  accurate,  are  termed  here  low-fidelity  (LF) 
models.  Combining  the  low-fidelity  models  with  more  accurate  but  expensive  high- 
fidelity  (HF)  models  can  provide  a good  combination  of  high  accuracy  and  low  cost. 

A common  approach  of  such  multifidelity  techniques  is  to  use  the  ratio  or 
difference  between  the  low-fidelity  and  high-fidelity  models  at  one  or  several  points  in 
order  to  correct  the  low-fidelity  model  at  other  points.  Recent  examples  of  such 
multifidelity  techniques  are  presented  in  Toropov  et  al.  (1999).  Haftka  (1991)  and  Chang 
et  al  (1993)  suggested  calculation  of  the  ratio  and  its  derivatives  at  one  point  in  order  to 
construct  a linear  approximation  of  the  ratio  at  other  points  in  design  space.  Fitting  the 
ratio  or  the  difference  as  a response  surface  approximation  has  the  advantage  over 
derivative  based  approximation  that  it  can  apply  over  larger  regions  in  the  design  space 
and  also  smooth  out  numerical  noise. 

The  correction  response  surface  approach  was  used  for  combining  high-fidelity 
Euler  solutions  with  lower-fidelity  panel  solution  for  the  design  optimization  of  the  high 
speed  civil  transport  (HSCT)  by  Knill  et  al.  (1999).  Mason  et  al.  (1994)  in  a different 
study  on  the  HSCT  instead  combined  lower-fidelity  2-D  finite  element  (FE)  models,  and 
higher-fidelity  3-D  FE.  Balabanov  et  al.  (1998)  combined  coarse  and  refined  FE  models 
to  predict  the  optimum  wing  bending  material  weight  of  the  FISCT  as  a function  of  the 
aircraft  geometry.  Kaufman  et  al.  (1996)  used  a different  strategy  to  combine  low-fidelity 
and  high-fidelity  models  for  the  optimization  of  the  takeoff  gross  weight  for  the  F1CST. 
They  used  inexpensive  constraints  to  eliminate  from  consideration  large  portions  of  the 
design  box  defined  by  the  upper  and  lower  limits  on  the  design  variables.  Response 


114 


surface  approximations  were  then  constructed  in  the  reduced  design  domain  based  on 
high-fidelity  methods  only.  Vi  tali  et  al.  (1997)  combined  strength-of-materials  models 
with  STAGS  FE  models  to  predict  stresses  and  buckling  loads  in  a hat-stiffened  panel. 
Vital  i et  al.  ( 1 998 ) and  Vitali  et  al.  ( 1 999)  demonstrated  the  use  of  a correction  response 
surface  for  calculating  stress  intensity  factors  for  cracks  in  stiffened  panels.  The  present 
work  summarizes  this  earlier  work  and  extends  it  in  that  response  surfaces  also 
approximate  the  low-fidelity  model.  The  high-fidelity  model  describes  the  crack  in  the 
stiffened  panel  with  a detailed  finite  element  model  near  the  crack  and  computes  the 
stress  intensity  factor  using  classical  linear  elastic  fracture  mechanics.  The  low-fidelity 
model  uses  a coarse  finite  element  model  that  does  not  model  the  crack.  The  stress 
intensity  factor  is  calculated  from  a closed-form  solution. 

In  one  approach,  a linear  correction  response  surface  is  fitted  to  the  values  ol  the 
ratio  and  the  difference  of  the  high-fidelity  and  low-fidelity  stress  intensity  factors  and 
used  with  a quartic  response  surface  based  on  the  low-fidelity  analyses.  In  a second 
approach,  quartic  response  surface  approximations  are  constructed  based  on  the  ratio  and 
difference  approximations  for  the  corrected  low-fidelity  stress  intensity  factors  in  terms 
of  the  design  variables.  The  quartic  response  surface  approximation  is  used  as  a 
constraint  in  the  design  optimization  problem. 

A comparison  is  also  carried  out  between  the  results  obtained  by  using  the 
correction  response  surface  approach  and  a single-fidelity  response  surface  approach. 

Correction  Response  Surface 

Correction  response  surfaces  techniques  couple  high-fidelity  and  low-fidelity 
methods  of  analysis  at  some  points  in  the  design  domain.  In  particular  two  ways  of 
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coupling  the  two  methods  of  analysis  are  considered  in  this  paper:  the  ratio  and  the 
difference. 

The  process  starts  with  high-fidelity  and  the  low-fidelity  computation  of  the 
response  of  interest,  y,  at  n design  points.  The  response  y is  a function  of  the  vector  of 
design  variables  x that  is  y = y(x).  At  the  n design  points  the  ratio  J3 

P = ^ (5.1) 

y lf 


and  the  difference  8 


8 = y hf  — y lf  (5-2) 

are  then  computed.  The  subscript  HF  and  LF  in  Eqs.  (5.1)  and  (5.2)  indicate  the  value  ot 
y obtained  from  a high-fidelity  and  a low-fidelity  analyses,  respectively.  Using  the  n 
values  of  /?  and  8,  two  response  surfaces  approximations  can  be  obtained  as  a function  of 
the  design  variable  vector  x,  that  is 

pres  = J3res(x ) and  8res  = 8res(x)  (5.3) 

The  multifidelity  approximation  to  at  any  other  point  is  obtained  from  a low- 
fidelity  analysis, yLF  as 

y = Pres  00  yLF  °r  y = 8m(x)  + yLF  (5.4) 

Implementing,  the  expressions  of  Eq.  (5.3)  and  Eq.  (5.4)  for  design  optimization 
requires  coupling  of  an  optimizer  with  the  low-fidelity  analysis  program.  Every  time  the 
optimizer  needs  a value  of  y,  it  has  to  call  the  low-fidelity  analysis  program,  and  correct 
the  low-fidelity  solution  with  the  correction  response  surface  (J3res  or  £res). 

In  order  to  avoid  the  need  to  interface  the  low-fidelity  analysis  and  the 
optimization  software,  in  this  chapter  a different  approach  was  investigated.  The  low- 
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fidelity  analysis  is  replaced  with  a high  order  polynomial  response  surface 
approximation.  The  optimizer  then  calls  the  response  surface  approximation  instead  of 
calling  the  low-fidelity  analysis  program  every  time  a value  ofy  is  needed.  It  is  important 
to  point  out  that  the  response  surface  approximation  could  be  more  accurate  than  the  low- 
fidelity  analysis  itself  as  response  surfaces  tend  to  filter  out  the  inherent  numerical  noise 
present  in  many  computer  simulations.  Using  the  response  surface  of  the  low-fidelity 

analyses,  y™  (x)  and  the  ratio  correction  response  surface  approximation  the  value  of y 

can  then  be  approximated  by 

y(x)  = r\x)y™(x)  (5.5) 

or  alternatively  using  the  difference  correction  response  surface  approximation  as 

y(x)  = 5n's  (x)  + y™  (x)  (5.6) 

In  addition  to  the  approximations  given  by  Eq.  (5.5)  and  Eq.  (5.6)  two  other 
approximate  expressions  of  y can  be  obtained  by  correcting  the  low-fidelity  values  yLF 
by  /?res  and  c>res  and  then  fitting  a high  order  polynomial  response  surface  approximation 
to  the  corrected  values  of  yIF . 

Example  Problem  Description 

The  dimensions  of  a composite  stiffened  panel  used  to  demonstrate  correction 
response  surface  techniques  are  shown  in  Figure  24.  The  composite  stiffened  panel  was 
subjected  to  a tensile  load  case  of  2000  lb/in. 
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20" 

Figure  24:  Wing  panel  geometry  with  a crack 


The  material  used  in  the  panel  was  AS4/3501-6  graphite/epoxy,  and  its  properties 
are  given  in  Table  12. 


Table  12:  Material  properties  of  AS4/3501  -6 


Material  property 

Value 

Young's  modulus  E| 
Young's  modulus  E2 
Shear  modulus  G12 
Poisson's  Ratio  V12 
Density  p 

Kq0  (Fracture  toughness) 

20  x 106  psi 
1 .4  x 106  psi 
0.76  x 106  psi 
0.30 

0.057  lb/in3 
100.000  psiVin 

The  geometry  of  the  panel  and  the  size  of  the  crack  (2a  = 4.0  inches),  as  well  as 
the  stacking  sequence  of  the  skin  and  the  stacking  sequence  of  the  stiffener  were  kept 
constant  throughout  the  study.  The  lay-up  used  for  both  the  skin  and  the  stiffener  ol  the 
panel  was  [+45/90/0]s.  The  thicknesses  of  different  plies,  present  in  the  skin  and  in  the 
stiffener  were  used  as  design  variables.  The  thicknesses  of  the  +45°  and  -45°  plies  in  each 
laminate  were  set  equal  to  each  other  to  maintain  balance.  A total  of  six  ply  thickness 
design  variables  were  used  and  were  labeled  as:  p4$,  pgo , poo,  b45,  bw,  boo • The  last  two 
digits  indicate  the  ply  angle  while  the  letter  “p”  and  the  letter  “b”,  respectively,  refer  to 
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the  plate  (skin)  or  the  blade  (stiffener).  All  design  variable  values  ranged  between  0.005 
inches  and  0.025  inches.  The  lower  limit  reflects  that  the  material  system  used  in  this 
study,  the  AS/3501-6,  is  generally  available  in  the  form  of  prepreg  tape  with  a nominal 
thickness  of  0.005  inch.  The  maximum  values  represent  estimates  of  the  required  material 
based  on  the  papers  by  Vaidyci  and  Sun  (1996)  and  Vaidya  et  al.  (1997). 

Finite  Element  Models 

The  stiffened  panel  is  simply  supported  along  the  edges  parallel  to  the  x-axis,  and 
free  along  the  edges  parallel  to  the  y-axis  (see  Figure  24).  A multipoint  constraint  was 
applied  simultaneously  to  the  skin  and  to  the  stiffener,  enforcing  the  same  displacement 
in  the  y-direction  along  the  simply  supported  edges.  Two  different  finite  element  models 
of  the  stiffened  panel  were  created  using  the  STAGS  program.  In  both  finite  element 
models,  symmetry  was  utilized  in  order  to  model  only  % of  the  structure,  thereby 
reducing  the  total  number  of  degrees  of  freedom. 

The  high-fidelity  STAGS  model  of  the  stiffened  panel  was  used  to  accurately 
capture  the  stress  gradients  in  the  region  near  the  crack  tip.  The  refined  model  consisted 
of  7,158  4-node  elements  and  46,680  degrees  of  freedom  and  is  shown  in  Figure  25. 


X 


Figure  25:  High-fidelity  finite  element  model  of  !4  of  wing  panel 
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The  low-fidelity  finite  element  model  was  coarser  and  it  did  not  contain  a crack. 
A uniform  mesh  of  4-noded  elements  for  a total  of  13284  degrees  of  freedom  was  used. 
The  model  was  used  to  compute  the  nominal  strains  and  stresses  in  the  location  where  a 
crack  is  assumed  to  be.  Then,  a closed  form  solution  was  used  to  compute  the 
approximate  stress  intensity  factor  in  the  presence  of  the  crack. 

Stress  Intensity  Factor  Calculation 

The  method  of  Vaidya  and  Sun  (1996)  was  used  to  predict  crack  propagation  in  a 
composite  laminate.  In  this  method  the  average  stress  intensity  factor,  K,  through  the 
thickness  of  the  plate  is  computed  by  modeling  the  laminate  as  an  equivalent  orthotropic 
plate.  Then  the  stress  intensity  factor  in  the  0°  ply  is  computed  using 

K°=rjK  (5.7) 

where  rj  is  defined  as  the  ratio  of  the  nominal  stress  in  the  0°  ply  to  the  average  stress  in 
laminate: 

<7° 

= — (5.8) 

ayy 

The  crack  in  the  skin  of  the  panel  is  assumed  to  propagate,  leading  to  structural 
failure,  when  stress  intensity  factor  in  the  0°  plies  K°  reaches  K°q  , the  fracture  toughness 

of  the  0°  ply.  The  value  of  the  fracture  toughness  K°q  was  taken  from  Vaidya  and  Sun 

(1996).  The  theoretical  distribution  of  <jyy  near  the  crack  tip  of  the  equivalent  orthotropic 
panel  is  given  by: 


K 


(5.9) 
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The  high-fidelity  value  of  K,  Khf,  was  then  obtained  by  least-square  fit  of  a>y 
from  FE  data  as  a function  of  the  distance  from  the  crack  tip,  r using  Eq.  (5.9). 
Calculation  of  one  Khf  required  about  700  seconds  of  wall  time  and  5 1 .7  seconds  of  CPU 
time  on  a 233  MHz  DEC  Alpha  Station  200  4/233. 

The  low-fidelity  stress  intensity  factor  Klf  was  calculated  using  the  low-fidelity 
FE  model  and  the  infinite  cracked  plate  solution  for  K 

K = ayy  Jna  (5.10) 

where  a)y  in  Eq.  (5.10)  is  the  average  stress  in  an  uncracked  equivalent  orthotropic  panel 
at  the  location  where  the  crack  is  assumed  to  be  present.  Calculation  of  one  value  of  Klf 
required  an  average  of  1 1 seconds  of  wall  time  and  2.6  seconds  of  CPU  time  on  a 233 
MHz  DEC  Alpha  Station  200  4/233.  The  low-fidelity  value  of  K obtained  is  approximate 
mainly  for  two  reasons:  (i)  the  finite  size  of  the  panel  was  not  taken  into  consideration  in 
Eq.  (5. 1 0)  and  (ii)  the  beneficial  effect  of  the  stiffener  on  reducing  the  stress  intensity 
factor  could  not  be  fully  captured  by  this  model. 

Linear  Correction  Response  Surfaces  Approximations 
The  high-fidelity  and  low-fidelity  methods  of  analysis  were  coupled  together  to 

model  the  crack  propagation  constraint.  The  motivation  for  coupling  the  two  analysis 
methods  was  to  try  to  achieve  the  accuracy  of  the  high-fidelity  analysis  with  the  low 
computational  cost  of  the  low-fidelity  analysis. 

The  original  six  design  variables  P45,  P90,  poo,  b^5,  b%,  boo  were  coded  so  that  they 
ranged  from  -1  to  +1.  The  coded  design  variables  that  correspond  to  the  original  design 
variables  were  named  P45,  P90,  Poo,  B45,  P90,  and  Boo ■ A 2(6"21  point  fractional  factorial 
design  in  the  6 coded  design  variables  was  selected  using  the  JMP  program  for  a total  of 
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16  design  points.  At  these  16  design  points,  the  value  of  the  stress  intensity  factor  K°  was 


calculated  using  both  the  high-fidelity  method  of  analysis  KQHF  and  the  low-fidelity 


methods  of  analysis  K°LF . Based  on  the  values  of  KFF  and  K"LF  the  ratio 


o 


0 


kr° 

P = HF 


KF 


(5.11) 


and  the  difference 


S = K°HF-K°LF  (5.12) 

was  computed  at  all  16  design  points.  The  16  values  of  the  ratio  /?  varied  between  0.86 
and  1.01  indicating  moderate  differences  between  the  high-fidelity  and  the  low-fidelity 
model.  The  16  values  of  the  difference  c>  varied  between  17.06  psiVin  and  16811  psi  Vin 

for  bladed  stiffened  panels  having  a high-fidelity  stress  intensity  factor  K°HFof  67106 
psiVin  and  109510  psiVin  , respectively.  The  largest  discrepancies  for  the  ratio  (3  and  the 
difference  8 among  the  16  design  points  did  not  occur  at  the  same  design  points,  however 
they  both  occurred  at  design  points  that  corresponded  to  bladed  stiffened  panels  with  a 
very  thin  skin  and  a very  thick  stiffener. 

One  linear  response  surface  approximation  for  the  ratio  and  one  linear  response  surface 
approximation  for  the  difference,  /?Lm  and  dLm  respectively,  were  obtained  fitting  the 
values  1 6 values  of  [i  and  8 as  a function  of  P45,  P90,  Poo,  P45,  B90,  and  Boo • The 
expressions  for  and  5 Lm  found  from  a least  square  fit  on  the  16  values  of  /?  and  8 


were 


=0.01  (93.7  + 2.39  P45  +0.612  ^ + 1 .48  P00  + 
-1.63  P45  - 0.479  P90  -2.21  P00) 


(5.13) 
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and 


8Lm  = -7009  + 3620  P45  + 1 108  P90  + 3856  P00  + H 

- 1455  P45  - 337  P90  - 1344  P00 

Once  the  expressions  for  the  correction  response  surface  approximation  were 
obtained,  the  stress  intensity  factor  in  the  0°  plies  for  any  other  design  can  estimated  by. 

K°  = pLm  K%  (5.15) 

or  by 

K°=SLin+K°LF  (5.16) 

An  additional  linear  response  surface  approximation  based  only  on  the  high- 
fidelity  stress  intensity  factors  of  the  previous  1 6 design  points  was  constructed  and 

named  K^Fm 


K%;m  = 104  (9.68  - 1.67  P45  -0.586  P90  - 3.53  P00  + 

-1.19  P45  - 0.336  P90  - 1.80  P00) 

The  accuracy  of  the  fit  was  checked  by  the  statistical  measures  presented  in 
Chapter  3 at  48  additional  high-fidelity  analyzes  at  the  points  that  complement  the  design 
to  a full  two  level  factorial  design. 

The  important  statistical  parameters  of  f3iAn , 5 L,n  and  K°HF"'  are  presented  in 

Table  13.  The  %RMSE0 r in  Table  13  is  defined  by  Eq.  (3.31)  with  % = 9 while 
%RMSEch  is  again  defined  by  Eq.  (3.31)  with  x = 48.  Table  13  also  reports  the  average 
values  of  the  stress  intensity  factor  found  for  each  different  approximations  and  the 
average  error  of  each  different  approximation.  As  can  be  seen  from  Table  13  the  most 
accurate  approximation  is  given  by  /?l and  the  least  accurate  one  is  given  by  K°HFm  The 


statistical  parameters  of  K°HFm  indicate  that  the  response  surface  is  not  accurate.  This  is 
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confirmed  by  the  optimization  problem  results  presented  later  where  K0H’^'n  was  used  as 
crack  propagation  constraint. 


Table  13:  Accuracy  measures  of  the  correction  response  surfaces  (3  Lm,  5 Lm  response 
surface  obtained  using  only  high-fidelity  analyses  The  subscript  OR  and  CH 


denote  the  original  set  of  1 6 points  and  the  independent  set  of  48  check  points, 

respectively 


Parameter 

(3  Lin 

5 Lin 

ts  0,  Lin 
^ HF 

R2 

0.954 

0.897 

0.814 

R Adj 

0.923 

0.830 

0.701 

RMSE 

1.16  10‘3 

2420 

2.82  103 

%RMSEor( Eq.  8)  y=16 

1.240 

35.10 

29.10 

% RMS  Eqw  (Eq.  8)  x=48 

0.858 

37.10 

18.50 

^ Approximate  /OR 

96729 

96842 

96843 

Av.  (Error)oR  of  K° 

734 

1503 

17514 

. ( K Approximate  )cH 

95524 

95662 

95107 

Av.  (Error)cH 

715 

2150 

14460 

Smallest  tslat 

1 .670 

0.750 

0.520 

p-value 

0.129 

0.460 

0.616 

The  high  p-values  for  the  smallest  tslat  of  both  SLm  and  Kj£ 

indicate  that  they 

have  a term,  which  turns  out  to  be  B90,  that  has  a high  probability  of  being  equal  to  zero. 
Removing  the  B90  term  from  the  expressions  of  <7Lm  and  K°Hfn  however  did  not  increase 
the  quality  of  the  approximations  and  therefore  the  term  was  kept  in  the  response  surface 
expressions  of  Slm  and  K^lFm  . 


Response  surface  approximation  based  on  the  low-fidelity  model 
Combining  the  correction  response  surfaces  /?Lm  and  Sl 'm  and  the  values  of  the 

stress  intensity  factor  obtained  from  the  low-fidelity  model  K°u,  provides  an  accurate 

approximation  to  the  crack  propagation  constraint.  However,  this  combination  still 
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requires  coupling  of  the  optimizer  with  the  low-fidelity  analysis  program  as  explained  in 
the  section  “Correction  response  surface”.  In  order  to  eliminate  this  requirement,  the  low- 
fidelity  analyses  were  approximated  by  with  a fourth  order  polynomial  response  surface.. 

The  400  design  points  were  chosen  using  the  latin  hypercube  (LHS)  design  in  the 
six  coded  design  variables  P45,  P90,  Poo,  B45 , B90,  and  Boo  assuming  a uniform  distribution 
for  each  design  variable.  LHS  is  a space  filling  design  and  it  tends  to  space  design  points 
evenly,  letting  each  design  variable  take  a particular  value  only  once.  However,  response 
surfaces  based  on  LHS  designs  are  often  not  very  accurate  at  comers  of  the  design  space. 
Therefore  a full  two-level  factorial  design  was  added  to  the  LHS  design  for  a total  of  464 

design  points.  The  KaLF  was  computed  at  the  464  points  and  a quartic  response  surface  in 

the  six  coded  design  variables  was  obtained.  After  stepwise  regression  was  carried  out 
148  out  of  the  original  210  terms  were  retained  for  a Cp=  114.  The  statistical  parameters 

of  the  resulting  quartic  response  surface  K°Lfuar"c  for  the  low-fidelity  model  are  presented 
in  Table  14. 


Table  14:  Statistical  properties  of  quartic  response  surface  approximation  of  Kau(r“an,c 

obtained  using  464  low-fidelity  analyses.  The  subscript  OR  and  CH  denote  the  original 
set  of  464  points  and  the  independent  set  of  4032  check  points,  respectively 


Parameter 

jy  0 , Ouartic 

klf~ 

N of  Terms 

148 

On 

114 

R5 

0.999 

R Adj 

0.999 

RMSE 

365 

%RMSEok  x=462 

0.404 

%RSMEch  X=4032 

1.430 

Smallest  tstat 

1.330 

p value 

0.184 
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In  addition  Figure  26  shows  the  prediction  of  the  quartic  response  surface 
approximation  based  on  464  low-fidelity  analyses  compared  with  the  values  from  4032 
additional  low-fidelity  analysis.  The  4032  additional  designs  were  obtained  by 
subtracting  the  64  designs  that  constitute  a two-level  factorial  design  from  the  4096 
designs  that  complete  a four-level  factorial  design.  If  the  quartic  response  surface 
approximation  were  perfectly  accurate  the  values  represented  by  dots  in  Figure  26  would 
lie  on  the  main  diagonal.  The  figure  indicates  that  the  response  surface  approximation  is 
very  accurate,  allowing  us  to  use  with  confidence  the  quartic  response  surface 

approximation  K^9uanic  instead  of  the  low-fidelity  analysis  in  the  optimization  procedure. 


Figure  26:  Quartic  response  surface  approximation  of  the  stress  intensity  factor,  K°u'rua"K 

based  on  464  low-fidelity  analyses  versus  the  stress  intensity  factor  obtained  from  low- 

fidelity  analysis  Kj.f 


Stress  intensity  constraint  approximation 
Two  different  strategies  were  used  to  couple  the  correction  response  surfaces  p m 


and  8Un  with  the  464  low-fidelity  analyses  in  order  to  obtain  the  expression  of  the  stress 
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intensity  factor  to  use  in  the  optimization  procedure,  resulting  in  a total  of  four  different 


approximations  for  the  crack  propagation  constraint. 

The  first  expression  for  the  stress  intensity  factor  approximation,  £0,/i , was 


obtained  by  multiplying  K°Lfuar"c  obtained  in  the  previous  section  by  /?l  m,  that  is 


0,  Quart  ic 


0,  Quart  ic 


(5.19) 


• • • o s 

while  the  second  expression  for  the  stress  intensity  factor  constraint,  K was 


obtained  by  adding  K^'Fuanu  to  <5'Lm  as 


0,  Quart  ic 


0,  Quartic 


(5.19) 


In  addition  two  other  expressions  of  the  stress  intensity  factor  approximation  were 
obtained  by  correcting  the  464  K°LF  values  used  in  the  previous  section  to  fit 
^.Quartic  usjng  ^Lin  anc)  £Lm5  anc]  then  fitting  a quartic  response  surface  to  the  464 

corrected  values  of  the  stress  intensity  factor.  The  response  obtained  by  multiplying  the 
K"n  values  by  /?l  in  was  named  K^nic  and  for  the  response  obtained  from  adding  K°LF  to 

b’Lin  was  named  K^artlc . The  statistical  parameters  of  the  quartic  response  surfaces 

obtained,  K(^farlic  and  K^anw  , are  presented  in  Table  15. 
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Table  15:  Quartic  response  surface  approximations  statistical  parameters  of  K((jfarllc  and 
K?jd  , obtained  from  correcting  Klf,  with  p Lmand  5 Lm.  The  subscript  OR  denotes  the 


original  set  of  1 6 points. 


Parameter 

K0,p 

Quartic 

K°'s 

Ouartic 

N of  Terms 

151 

148 

R2 

116 

114 

0.999 

0.999 

R Adj 

0.999 

0.999 

RMSE 

341 

365 

% RMS  Eon 

0.400 

0.440 

Smallest  t_stat 

1.000 

0.090 

p value 

0.317 

0.929 

In  order  to  assess  in  a more  reliable  way  the  accuracy  of  the  four  expressions  of 
the  stress  intensity  factor  approximations  K0,p , KQ,S , Kguprlic  and  the  prediction 

of  the  approximations  were  compared  with  the  stress  intensity  factor  obtained  from  the 
high-fidelity  method  of  analysis  for  the  original  1 6 design  points  used  to  construct  the 


correction  response  surfaces  fiUn  and  and  for  the  additional  48  points.  The  results  of 
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the  comparison  are  presented  in  Table  16. 


Table  16:  Comparison  of  the  accuracy  of  the  stress  intensity  factor  constraints  obtained 
by  different  methods.  K^uprllc  is  the  quartic  approximation  obtained  from  464  /3  m 

corrected  values.  K^’fartlc  is  the  quartic  approximation  obtained  from  464  8Un  corrected 

values.  K°'p  is  the  approximation  obtained  by  multiplying  the  response  surface  J3l  1,1  by 

A O 

the  quartic  approximation  based  on  464  low-fidelity  analyses.  K ’ is  the  approximation 

I * 

obtained  by  adding  the  response  surface  (5  to  the  quartic  approximation  based  on  464 
low-fidelity  analyses.  The  subscript  OR  and  CH  denote  the  original  set  of  1 6 points  and 
the  independent  set  of  48  check  points,  respectively. 


Parameter 

K0,p 

Quartic 

K0,s 

Quartic 

K°’p 

K°'s 

%RMSEor  x=16 

1.68 

3.67 

1.68 

2.82 

%RMSECn  x=48 

1.11 

3.29 

1.11 

2.67 

%Err  Max  or  x=16 

1.39 

6.08 

1.41 

6.58 

%Err  Maxc  h x=48 

1.98 

5.31 

1.96 

4.81 
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As  can  be  seen  from  Table  16  the  four  approximations  K0’^ , K0,s , K^farllc  and 

C\  C 

K0uarllc  differ  in  their  accuracy.  In  particular  it  appears  that  the  most  accurate 
approximation  is  given  by  the  approximation  based  on  fiLm,  K0,/i  However  from  the 

results  shown  in  the  first  four  columns  of  Table  16,  it  appears  that  there  is  no  significant 
difference  in  the  prediction  of  the  stress  intensity  factor  approximations  if  the  correction 

response  surfaces  are  multiplied  or  added  (K°'d)  to  the  quartic  response  surface 

based  on  low-fidelity  analyzes  or  if  the  low-fidelity  data  are  first  corrected  and  then  a 
quartic  response  surface  is  fitted  to  the  corrected  data  ( A^;frI/c.  and  K^artw ). 

Response  surface  approximation  based  only  on  high-fidelity  method 
Coupling  the  high-fidelity  method  and  the  low-fidelity  method  of  analysis  had  a 

computational  cost.  In  particular  the  total  amount  of  CPU  time  required  by  16  high- 
fidelity  analysis  and  464  low-fidelity  analysis  was  about  2080  sec.  The  same  amount  of 
CPU  time  is  required  to  perform  about  40  high-fidelity  analyses  (if  wall  time  is 
considered,  than  only  24  high-fidelity  analyses  can  be  performed)  consequently  40  high- 
fidelity  analyses  were  carried  out  for  comparison.  Sixteen  high-fidelity  analysis  were 
already  available  from  the  construction  of  the  correction  response  surfaces  /?Lm  and  8Un. 
Thirteen  additional  points  were  selected  by  choosing  the  design  with  all  coded  variable 
set  to  zero  and  then  letting  each  design  variable  to  take  the  values  +1  and  -1 . Finally,  1 1 
D-optimal  points  were  chosen  from  the  48  points  that  with  the  original  1 6 complete  a two 


level  factorial  design. 
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A quadratic  response  surface  in  the  six  coded  design  variables  P45,  P 90,  P 00,  B45, 
B90,  and  Boo,  called  K®u°draUc , was  fitted  to  the  high-fidelity  results.  Table  17  presents  the 

most  significant  statistical  parameters  of  K%adrauc , after  the  stepwise  regression 
procedure  based  on  Cp  was  applied. 


Table  17:  Response  surface  approximation  Kfp,ad,a"L  obtained  from  only  high-fidelity 

analyses.  The  subscript  OR  and  CH  denote  the  original  set  of  40  points  and  the 

independent  set  of  37  check  points,  respectively 

Parameter  K2ufdralic 


N of  Terms 

19 

Cp 

11.76 

R5 

0.990 

R Adj 

0.980 

RMSE 

6137 

Smallest  t_stat 

1.420 

p value 

0.170 

%RMSEor  x=40 

6.460 

%RMSECh  X=37 

12.97 

%Err_Max or  x=40 

9.070 

%Err  Max ch  X=37 

40.77 

The  prediction  capabilities  of  K% ‘fra"c  can  compared  with  the  other  forms  of 
approximations  K0,fl , K°’s , K^rllc  and  K^artic  shown  in  Table  16.  In  Table  18  the 

values  of  KQ™dm,c  labeled  by  “OR”  refer  to  the  40  points  used  to  construct  the  quadratic 

response  surface  approximation  while  the  quantities  labeled  “CH”  refer  to  the  37  points 
that  with  29  out  of  the  40  original  points  complete  the  two  level  factorial  design. 
Comparing  the  results  presented  in  Table  16  and  Table  1 8 it  is  clear  that  the  response 

surface  approximation  K(^hadra,lc  is  not  as  accurate  as  the  approximations  obtained  using 


the  correction  response  surface  approach. 


130 


Optimization 

The  minimum  weight  optimization  formulation  for  the  composite  stiffened  panel 
shown  in  Figure  25  is  summarized  in  Table  18. 


Table  18:  Minimum  weight  optimization  problem  for  the  bladed  stiffened  panel 


Objective  Function: 

MinjWeight} 

Design  Variables  : 

P45,  P90,  Poo,  B45,  B90,  Boo 

Constraints: 

-1  < P45  < +1 

-1  < P90  < +1 

1 

A 

0 

0 

A 

+ 
»— * 

-1  < B45  < +1 

-1  < B90  < +1 

-1  < Boo  < +1 

K°<  100,000  psi  Vin 

The  weight  optimization  problem  was  implemented  into  the  Excel  ( Microsoft 
Corporation,  1998)  solver  tool  and  repeated  for  each  of  the  six  approximations  of  the 


stress  intensity  factor  K°’p , K°’s , K0o’farUc , K^fartic , K and  Kr“adra“c . The  minimum 


weights  obtained  are  summarized  in  Table  19. 
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Table  19:  Optimum  designs  obtained  from  the  different  stress  intensity  factor 
approximations.  %Err  denotes  the  percentage  error  of  each  approximation  at  its  optimum 

(negative  when  conservative) 


Parameter 

K°’P 

Ouartic 

K0,s 

Ouartic 

K°’fi 

K°-s 

ts  Quadratic 

khf 

IS  L in 

khf 

P45 

-1 

-1 

-1 

-1 

-1 

-1 

P90 

-1 

-1 

-1 

-1 

-1 

-1 

Poo 

-0.290 

-0.333- 

-0.326 

-0.355  -0.194 

0.481 

B45 

-1 

-1 

-1 

-1 

-1 

-1 

B90 

-1 

-1 

-1 

-1 

-1 

-1 

Boo 

1 

1 

1 

1 

1 

1 

%Err 

1.890 

0.260 

0.300 

-0.880  6.270 

34.5 

Weight  (lb) 

0.446 

0.441 

0.442 

0.438 

0.457 

0.533 

Moreover  the  optimizer  always  found  the  minimum  weight  structure  by  setting  all 
the  non  zero  degree  plies  to  the  minimum  thickness  gage  and  then  increasing  the 
thickness  of  the  0°  plies  in  the  blade  as  much  as  possible.  Once  the  upper  bound  on  Boo 
was  reached  the  optimizer  started  increasing  the  0°  plies  in  the  panel,  Poo- 

The  most  accurate  values  at  the  optimum  are  obtained  for  the  approximations 

based  on  Sl However  all  four  approximations,  K0,p , K0,d  , K‘>hfirllc  and  K^’farlic  are  very 

accurate  and  are  in  line  with  the  results  presented  in  Table  16.  In  addition  it  can  be  also 
noticed  how  the  approximations  based  only  on  a single-fidelity  methods  and 

K'~7dra"c  are  less  accurate  than  the  multiple-fidelity  methods. 


Concluding  Remarks 

In  many  engineering  design  problems  optimization  presents  the  obstacle  of 
requiring  computationally  expensive  analyses.  In  order  to  overcome  this  difficulty  more 
attention  has  been  recently  given  to  simple  models  that  were  used  as  design  tools  a 
generation  ago.  These  models  are  often  referred  to  as  low-fidelity  models  and  though  not 
very  accurate  in  nature  they  are  computationally  inexpensive.  This  study  demonstrates 
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that  combining  the  low-fidelity  models  with  more  accurate  but  expensive  high-fidelity 
models  it  is  possible  to  achieve  high  accuracy  at  a low  computational  cost. 

A crack  propagation  constraint  for  composite  stiffened  panel  designs  is  used  as  an 
example  to  demonstrate  the  feasibility  of  combining  low-fidelity  models  and  high-fidelity 
models  to  obtain  accurate  results  at  a low  computational  cost.  Two  linear  polynomial 
correction  response  surface  approximations  were  fitted  to  the  ratio  and  the  difference  of 
the  stress  intensity  factors  obtained  from  a high-fidelity  model  and  a low-fidelity  model 
at  a relatively  small  set  of  design  points.  Two  different  approaches  were  then  used  to 
combine  the  low-fidelity  and  high-fidelity  methods.  The  first  approach  used  the 
correction  response  surfaces  together  with  a quartic  response  surface  that  approximated 
the  low-fidelity  analyses.  The  second  approach  instead  corrected  the  values  of  the  low- 
fidelity  analyses  and  then  fit  a quartic  response  surface  to  the  corrected  values.  The 
results  obtained  using  the  correction  response  surface  approach  proved  to  be  more 
accurate  than  the  results  obtained  using  only  high-fidelity  analyses  requiring  the  same 
amount  of  computational  resources. 

In  addition  there  was  not  much  difference  if  the  ratio  or  the  difference  was  used  to 
approximate  the  crack  propagation  constraint.  Moreover  it  did  not  matter  if  the  low- 
fidelity  analyzes  were  fitted  with  a quartic  response  surface  and  then  coupled  with  the 
correction  response  surface  or  if  the  low-fidelity  analyses  were  first  corrected  and  then 
fitted  with  a quartic  response  surface  approximations. 


CHAPTER  6 

RESPONSE  SURFACE  APPROXIMATIONS  INCLUDING  GRADIENT 

INFORMATION 


Introduction 

In  the  past  decade,  there  has  been  growing  interest  in  using  response  surface 
approximations  with  data  generated  by  computer  simulation,  partly  due  to  the  realization 
that  these  simulations  often  have  substantial  numerical  noise  that  can  be  reduced  by  the 
noise  filtering  properties  of  response  surface  approximations.  In  computer  simulations, 
derivatives  are  often  available  and  frequently  at  a lower  computational  cost.  As  a 
consequence,  there  is  an  interest  in  considering  response  surface  approximations  that 
incorporate  both  function  and  derivative  data.  Unfortunately  it  is  not  possible  to  apply  the 
same  methods  used  for  the  traditional  response  surface  methodology  since  function  and 
gradient  values  are  strongly  correlated.  As  a result,  any  methodology  that  attempts  to  use 
function  and  gradient  information  should  take  into  consideration  the  strong  coupling 
existing  between  the  two  types  of  data. 

A similar  situation  of  strongly  correlated  forms  of  data  is  encountered  in  the 
analysis  of  multiresponse  experiments.  An  experiment  in  which  a number  of  responses 
are  measured  simultaneously,  for  each  setting  of  a group  of  input  variables,  is  called  a 
multiresponse  experiment.  The  analysis  of  data  from  a multiresponse  experiment  requires 
careful  consideration  of  the  multivariate  nature  of  the  data,  and  as  a consequence,  the 
responses  should  not  be  investigated  individually  and  independently  of  each  other. 
Interrelationships  that  may  exist  among  responses  can  render  such  univariate 
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investigation  inaccurate.  Several  researchers  including  Box  et  al.  (1970),  Khuri  and 
Cornell  ( 1 996),  and  more  recently  Robinson  (2000)  investigated  the  problems  associated 
with  creating  a number  of  responses  simultaneously.  However  multiresponse 
methodology  cannot  be  applied  directly  to  approximations  based  on  function  and 
gradients  values.  When  using  function  and  gradients  of  a quantity  at  the  same  time, 
problems  may  arise  from  the  fact  that  not  only  there  are  several  functions  to  be  fitted  at 
the  same  but  additionally  the  fitting  parameters  are  the  same.  In  other  words  the  design 
points  and  the  parameters  of  each  approximation  (function  or  gradient)  are  the  same 
while  the  shape  functions  that  define  the  function  and  the  gradient  approximation  are 
different. 

Etman  (1992),  and  subsequently  Van  Keulen  el  al.  (2000),  used  function  and 
gradient  information  to  obtain  response  surface  approximations.  However  in  thier  studies 
both  Etaman  and  Van  Keulen  et  al.  had  little  or  no  control  on  the  noise  level  in  the  data, 
and  thus  did  not  know  the  form  of  the  true  function.  In  this  chapter  an  attempt  is  made  to 
show  the  effectiveness  of  inclusion  of  gradient  information  in  creating  response  surface 
approximations.  Simple  numerical  problems  were  generated  maintaining  good  control  on 
the  coupling  between  function  and  gradient  values,  noise  level,  and  the  form  of  the  true 
function  that  it  is  approximated. 

Regression  analysis  based  on  function  and  gradient  values 

Assume  that  y denotes  the  function  that  we  want  to  approximate,  and  that  it  is  a 

function  of  the  design  variables  x with  x € 91”*  where  njv  indicates  the  number  of  design 
variables,  thus  y = y>(x).  The  derivative  ofy  corresponding  to  the  direction  d,  where 
|d||  = 1,  is  denoted  by  yd  and  is  determined  by 
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yd  = (grcid'y)  d (6.1) 

Consider  n design  points  x;  (i  = 1 ..n)  where  for  each  design  point  the  function 
value  y(\\)  and  the  y directional  derivatives  exist.  The  associated  derivative  directions  are 
denoted  by  c/y  (i=l  ..n,  j=l  .Mdv)  and  a compact  notation  can  be  introduced  so  that  function 
values  and  the  derivatives  can  be  denoted  by  ytJ.  The  components  yv  are  defined  as 

To  =T,  =T(X,)>  i = \-n 

(6.2) 

y,j  = [grad  y]x=x  dtJ , j = \..n  and  j = \..y 

Obviously  if  y = 0 then  the  common  case  of  response  surface  approximation  based 
on  only  function  values  is  obtained.  The  previous  notation  can  be  compacted  even  further 
by  denoting 

y,7  =[ylQ-yir]  (6-3) 

In  principle,  the  length  of  may  vary  from  point  to  point,  as  y is  not  necessarily 
constant  for  all  points.  In  this  work  however  the  additional  restrictions  of  y = n<jv  and 
dij  = ej  with  e,  being  the  basis  vector  in  coordinate  directions,  are  imposed. 

The  response  function  y will  be  approximated  using  an  approximation  y defined 

by 

j>  = f(x)'P  (6.4) 


where  (3  is  the  vector  of  unknown  parameters  and  f(x)  is  the  vector  of  shape  functions, 
usually  monomials  of  polynomials.  Directional  derivatives  follow  as 

ytJ=  grP  (6-5) 

with 


g = g(x;ey)  = (grad1  f(x))  e, 


(6.6) 
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Evaluation  of  the  response  surface  approximation  of  Eq.  (6.4)  and  its  derivatives 
of  Eq.  (6.5)  for  Xj,  and  the  associated  base  vector  e,  gives 

y = A,p  (6.7) 

where  v is  defined  in  an  analogous  way  to  Eq.  (6.3).  The  matrix  A,  is  defined  as 


IX  >8/1  >§/2 ] 


(6.8) 


with  f,  = f(x,)  and  g0  = g(x,;e/). 

The  response  surface  of  the  function  y,o  and  its  derivatives  yy  is 

V = V + £ 

S ij  S ij  1/ 


(6.9) 


where  &,o  refers  to  the  error  in  the  function  value,  and  £ij  for  j ^0  refers  to  the  errors  in  the 
directional  derivatives.  From  direct  generalization  of  the  notation  used,  Eq.  (6.9)  can  be 
written  in  the  familiar  form 


y,  =y,+£, 


(6.10) 


with  £j  being  defined  in  an  analogous  way  to  Eq.  (6.3). 

The  vector  of  regression  coefficients  P for  the  response  surface  approximation  of 

Eq.  (6.4)  is  determined  using  a weighted  least  square  fit.  Unlike  standard  weighted  least 
square  (WLS)  methods  in  this  study,  information  on  both  function  values  and  derivatives 
are  included.  The  weighting  is  introduced  into  the  least  square  procedure  by  defining  the 
inner  product  as 

a • b = aTW  b (6.11) 

where  in  Eq.  (6.1 1)  a and  b are  two  vectors  and  W is  the  weighting  matrix.  Thus,  the 
least  squares  formulation  is  governed  by 

¥ = YJ(y,  -y,Y  w,(y,  -y,) 

/=! 


(6.12) 
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The  estimates  b for  the  regression  coefficients  p are  selected  as  to  minimize  y/ 
Substitution  of  Eq.  (6.7)  into  Eq.  (6.12)  and  minimization  of  the  latter  gives 


n 


n 


b=  YAfW.A,  2>;w4y, 


(6.13) 


/ = ! 


k = \ 


In  order  to  achieve  a more  compact  notation,  the  following  notation  can  be 


introduced 


X'  = [A[,...,A7„] 


(6.14) 


y'  =[y^-,y'J 


T 


(6.15) 


/ 


W = 


W,  0 

0 vv. 


0 A 
0 


V 


0 0 •••  w. 


«/ 


(6.16) 


and 


ru/vi-i 


H = {X  WX} 


(6.17) 


With  these  definitions  Eq.  (6.13)  can  be  written  in  a compact  form  as 


b = HX1  Wy 


(6.18) 


Similarly  to  y,  y and  £ can  be  defined  as  in  Eq.  (6.1 5).  With  these  definitions  it  is 

possible  to  write  an  equation  that  has  the  familiar  look  of  Eq.  (3.21) 

y = Xp  + £ (6.19) 

It  should  be  noted  that  Eq.  (6.19)  also  includes  directional  derivatives  and 
therefore  is  fundamentally  different  from  Eq.  (3.21).  In  principle,  the  previous 
formulation  can  be  generalized  further  by  dropping  the  special  form  of  the  matrix  W in 
Eq.  (6.16)  and  by  considering  a fully  populated  matrix. 
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Statistical  properties  of  gradient  based  response  surfaces 
When  dealing  with  response  surface  approximation  based  only  on  function 

values,  it  is  customary  to  assume  that  the  data  satisfy  the  Gauss-Markov  conditions  or 


E{  e)  = 0 


(6.20) 


cov(e)  = a1  I 


(6.21) 


in  other  words  that  the  random  errors  have  zero  mean,  are  uncorrelated,  and  have  the 
same  variance  a . Moreover,  it  is  also  typically  assumed  that  errors  are  distributed 
normally.  Under  these  conditions,  the  least  squares  formulation  provides  the  best  linear 
unbiased  estimate  (BLUE)  as  in  Sen  and  Sr  iv  as  lav  a (1990).  When  gradient  information  is 
included,  the  Gauss-Markov  condition  of  Eq.  (6.2 1 ) must  be  relaxed  as 


cov(s)  = C 


(6.22) 


The  matrix  C is  assumed  to  be  symmetric  and  positive  definite.  It  is  further 
assumed  that  the  errors  corresponding  to  different  points  are  independent,  that  is 


rp  

E(  £,  g ) = 0 for  i ^ j so  that  C reduces  to  a form  similar  to  Eq.  (6.16)  that  is 


C = 


C,  0 

0 c. 


0 

0 


\ 


• • 


• • 


0 0 •••  c 


(6.23) 


n) 


with  Cj  = cov(  £,  z\ ) = E(  £,  z\  ).  The  assumption  on  the  normal  distribution  on  £ is  still 


retained.  As  E(  £ ) = 0,  it  follows  that 


£(y)  = xp 


(6.24) 


and 
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cov(y)  = C 

Under  these  assumptions  it  can  be  demonstrated  that 

E(  b)  = P 

and  the  covariance  of  the  parameters  b can  be  calculated  as 

cov(b)  - HXtWCWtXHt 

From  Eq.  (6.27)  it  follows  that  if  the  weighting  matrix  W is  chosen  as 

W = C ' 

the  covariance  of  the  parameters  b is  greatly  simplified  as 

cov(b)  = (XT<r'X)_1 


(6.25) 

(6.26) 

(6.27) 

(6.28) 
(6.29) 


It  can  be  demonstrated  that  the  vector  of  parameters  b obtained  using  the 
relationship  of  Eq.  (6.28)  is  again  BLUE  (Sen  and  Srivastava,  1990). 

From  cov(b)  it  is  possible  to  calculate  an  estimate  of  the  reliability  of  the  b 
coefficients  by  taking  the  ratio  between  the  value  of  a coefficient  b,  and  the  square  root  of 
its  variance  var(bt 


t 


st at , / 


(6.30) 


The  tstat  can  be  used  to  reduce  the  number  of  parameters  of  a response  surface 
approximation  by  eliminating  the  coefficient  with  the  lowest  ftat-  The  procedure  is 
repeated  until  all  remaining  coefficients  have  a tstat  that  is  higher  than  a specified  limit. 
The  coefficients  are  eliminated  one  at  the  time  because  when  one  coefficient  is  deleted 
the  tstat  of  all  the  other  coefficients  change.  The  limit  can  depend  on  the  individual 
problem  but  good  results  were  obtained  in  this  chapter  by  using  two  as  the  limit. 
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Iterative  procedure 

A further  simplification  in  the  structure  of  the  covariance  C can  be  made  by 
assuming  that  the  errors  at  different  design  points  are  uncorrelated  but  that  the  correlation 
between  function  and  gradients  at  each  design  point  is  the  same.  That  implies  that  the 

matrix  C of  Eq.  (6.23)  can  be  written  in  the  form 

C = I„®C  (6.31) 

where  n is  the  number  of  design  points  and  C is  the  function  value-gradient  covariance 
matrix.  The  subcovariance  C is  a symmetric  (1  +n(/v)  x (1+  n^v)  matrix  where  n(/v  is  the 

number  of  design  variables.  Thus,  C is  a nt  x nt  matrix  where  n,  is  the  total  number  of 
observations  available  (functions  + gradients),  that  is  nt  = n («<*+ 1). 

However,  the  covariance  matrix  C is  rarely  known  and  therefore  an  estimate  has 
to  be  used  to  achieve  a proper  weighting  to  minimize  the  cov(b)  as  in  Eq.  (6.29).  The 

process  of  finding  an  estimate  of  C leads  to  an  iterative  procedure  since  the  covariance 

matrix  C depends  on  the  vector  of  parameters  b.  However,  b depends  on  C through  the 
weighting  matrix  W = C’  used  in  the  WLS  procedure.  Substituting  Eq.  (6. 1 7)  into  Eq 
(6.18)  the  coefficients  b are  obtained  as 

b = (XT  WX)1  XT  Wy  (6.32) 

where  X is  the  model  matrix  of  Eq.  (6.14),  W is  the  weight  matrix,  and  y is  the  vector  of 
measurement  as  in  Eq.  (6.19).  If  there  is  no  a priori  information  available  about  the 
covariance  matrix,  the  identity  matrix  can  be  used  as  the  starting  weight  matrix  of  an 
iterative  procedure  that  attempts  to  estimate  C.  After  solving  the  least  squares  problem  of 
Eq.  (6.32),  b is  used  to  calculate  y : 


y = Xb 


(6.33) 
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and  subsequently  the  residual  vectors  r*  can  be  calculated  as 


ro  = 


u -y\ 


y„  - y„ 


and  r.  = 


d.Xj 


dx . 


dy„ 

dx . dx . 


for  y=l..  n 


dv 


(6.34) 


A biased  estimate  of  the  covariance  matrix,  which  is  the  best  estimate  obtainable 
under  the  present  assumptions,  can  be  calculated  ( Khuri  and  Cornell,  1 996  and  Zellner, 
1962) 


Using  W = I„® 


(6.35) 


n 

/V  . 

C as  in  Eq.  (6.32),  a new  set  of  coefficients  can  be  found 


which  is  subsequently  used  to  estimate  a new  covariance  matrix  C . The  procedure  is 
then  iterated  until  convergence  is  achieved  for  the  coefficients.  The  following  algorithm 
called  the  iterative  covariance  algorithm,  is  used  to  solve  the  iteratively  weighted  least 
squares  problem  and  is  defined  as  follows: 

i.  Assume  a form  for  the  error-correlation  C 

ii.  Solve  the  least  square  problem  with  initial  weight  matrix  W = I 

a)  Calculate  residual  vectors  r,,  j = 1 ,..,n  according  to  Eq  (6.34) 

/V 

b)  Estimate  C from  Eq.  (6.35) 

c)  Calculate  |b|  and  if  the  variation  of  jb|  is  less  than  a specified  small 
number  than  stop,  otherwise  return  to  a) 

iii.  Calculate  the  tstat  and  other  statistical  parameters 
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A 

Different  forms  of  the  estimated  covariance  submatrix  C 
If  the  approximating  function  y of  Eq.  (6.4)  used  is  a polynomial  of  order  P , then 

the  total  number  of  parameters  to  be  estimated  is  n\,  + nc  where 


„,=fi<"*+i) 


/=] 


i\ 


(6.36) 


is  the  number  of  coefficients  in  the  polynomial  approximating  functions  and 


n _y  •_(^v+])Kv+2) 
i=i  2 


(6.37) 


is  the  number  of  unknown  entries  in  the  estimated  covariance  submatrix  C . For  example, 
for  a cubic  polynomial  in  two  design  variables  there  are  a total  of  10  + 6 =16  terms  to  be 
estimated. 


In  general  the  component  C of  the  covariance  matrix  as  in  Eq.  (6.31),  may  be 
written  as 

where  C}J  = Cp  (6.38) 

where  subscript  ‘O'  indicates  function  values  while  subscripts  [1  denote  derivatives. 

th 

For  example  Cos  is  the  correlation  between  function  values  and  the  derivatives  of  the  5 
design  variable. 

Based  on  the  physics  of  the  problem  or  on  experience,  an  a priori  knowledge  ol 

A 

the  structure  of  C might  make  it  possible  to  reduce  the  number  of  unknowns,  while  at  the 
same  time  achieving  better  accuracy.  One  simpler  form  corresponds  to  all  derivatives 
having  the  same  correlation 


C C 

MO  ^ 


01 


C 


On 


civ 


c 


c c 

Mo  Ml 


c 


1 nch 


c c 

Mo  Mo 


c 


^ih’^ch 
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r r ■ C 

V_-  nn  '-'l 


C = 


00  '-'1 

Ct  C, 


c 


(6.39) 


c,  a •••  c. 


An  even  simpler  form  arises  when  derivatives  and  functions  are  not  correlated, 

A 

but  have  different  variances.  In  this  case  the  resulting  covariance  submatrix  C is  a 
diagonal  matrix 


C = 


c 


00 

0 


0 

c 


II 


0 

0 


0 0 •••  c„ 


(6.40) 


A special  case  of  the  covariance  matrix  is  found  when  all  gradients  have  the  same 


variance 


C = 


C’  0 


oo 

0 


c 


II 


0 

0 


0 0 


c 


(6.41) 


* 1 

Finally,  the  greatest  simplification  possible  is  C = a~ I„^+l , that  is  the  variance  of 
the  errors  in  the  function  and  in  the  gradients  are  the  same,  and  no  correlation  exists 


between  them. 


Numerical  tests 

Numerical  examples  of  approximations  obtained  from  function  and  gradient  data 
are  presented  and  discussed.  The  response  surface  approximations  obtained  by  using 
function  and  gradient  information  were  compared  with  approximations  obtained  by  using 
function  values  only.  The  following  numerical  examples  attempt  to  find  evidence  of  the 
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advantages  and  disadvantages  of  including  gradient  information  in  the  fitting  procedure 
used  to  calculate  the  parameters  of  a response  surface  approximation. 

Modeling  or  Bias  Errors 

This  section  is  focused  on  modeling  (bias)  error,  that  is  when  the  approximating 
function  has  a different  form  then  the  true  function.  It  is  assumed  that  the  number  of 
design  points  where  function  and  gradient  information  are  available  is  fixed,  as  may 
happen  when  availability  of  computational  resources  is  limited.  Consider  the  case  of 
when  the  number  of  function  values,  n,  is  sufficient  to  obtain  a polynomial  of  order  P.  If 
gradients  are  available,  the  total  amount  of  data  available  increases  from  n to  n {ridv+ 1). 

The  benefits  of  using  gradients  to  improve  the  quality  of  an  approximation  is 
demonstrated  by  the  following  two  example  problems  where  a quadratic  response  surface 
approximation  is  used.  In  the  first  example  the  true  function  and  the  approximating 
function  are  function  of  two  design  variables  while  in  the  second  example  the  true 
functions  and  the  approximations  are  function  of  ten  design  variables.  The  two  examples 
presented  consist  of  functions  of  variables  that  range  from  - 1 to  + 1 . 

The  form  of  the  response  surface  approximation  sought  in  the  first  example  is  a 
quadratic  polynomial  in  the  two  variables  Xj  and  .w 

y = (x,,x2)  = b0  + blxi  + b2x2  + b2x]  + 64x,x2  + bsx2  (6.42) 

Approximations  based  only  on  function  values  obtained  from  a 3-level  factorial 
design,  for  a total  of  nine  points,  are  compared  to  the  approximations  obtained  with 
gradient  and  function  values  at  the  same  design  points  for  a total  of  27  data. 
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th 

Example  1 . Quadratic  approximation  of  a 4 order  polynomial 

Consider  the  case  where  the  true  function  is  the  following  full  quartic  polynomial 

in  two  variables: 


/(x,,x2)  = 1 + x,  + x2  + x]  + x,x2  + x2  + 
+ x,1  + x2x2  + x,x2  + x2  + 

4 1 2 2 3 4 

+ X,  + X,Xj  + XJX,  + x2x2  + x2 


(6.43) 


and  two  quadratic  polynomials  approximations  are  created: 

• y , approximation  obtained  using  function  and  gradients  at  nine  design 


points 


yf  approximation  obtained  using  only  function  values  at  the  same  nine 


design  points  used  to  obtain  y . 

The  covariance  and  weight  submatrices  for  y after  six  iterations  of  the  iterative 
covariance  algorithm  are: 


1.749 

5.132 

5.132" 

"13.552 

-2.212 

-2.212" 

5.132 

16.764 

14.669 

and  W = I9  0 

-2.212 

0.616 

0.138 

(6.44) 

5.132 

14.669 

16.764 

-2.212 

0.138 

0.616 

Note  that  the  resulting  approximate  correlation  matrix  shows  high  correlation 
between  function  values  and  gradients. 

Figure  27  shows  the  two  response  surfaces  y and  y f along  with  the  derivatives 
with  respect  to  x.  The  derivatives  X2  are  not  shown  since  they  are  identical  to  X/ 
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“i  ^ 

c)  d) 


Figure  27:  Response  surfaces  approximations  and  corresponding  derivatives.  Dots 
represent  the  data  used  a)  response  surface  obtained  from  function  values  only  yj  b) 

derivative  of  yf  c)  response  surface  obtained  from  function  values  and  gradient  data  yg 

d)  derivative  of  y 

o 


Table  20  presents  the  value  of  the  coefficients  obtained  for  yg  and  yf  along  with 

the  values  of  the  RMSE  based  on  function  values  and  gradient  values.  It  should  be  noted 
that  the  RMSE  presented  in  Table  20  is  calculated  from 

RMSE  = — (6.45) 

V n 

where  SSE  can  be  the  sum  of  the  square  errors  on  the  function  or  on  the  gradients 
depending  on  the  case  examined.  From  the  RMSE  values  reported  in  Table  20,  it  appears 
that  the  approximation  based  only  on  function  values  is  more  accurate.  However  when 
the  RMSE  was  checked  from  the  analytical  expression,  called  Re,  yg  resulted  in  a more 

accurate  approximation  than  yf  in  both  function  values  and  gradient  values.  In  addition 
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Table  20  presents  the  approximation  yMmHias  obtained  by  minimizing  the  RMSE  over  the 

entire  design  domain  from  it  analytical  formulation  and  the  approximation  y,  obtained 
using  function  and  gradient  data  but  using  the  Identity  matrix  as  a weighting  matrix. 


Table  20:  Parameters  and  error  measures  of  the  approximation  yMmB,as  that  minimizes 
analytically  the  bias  error,  of  the  approximation  yg , obtained  using  function  and  gradient 

values  with  iterative  covariance  algorithm,  of  the  approximation  y,  obtained  using 
function  and  gradient  values  with  an  identity  matrix  as  a weighting  matrix  and  of  the 
approximation  yf  obtained  using  only  function  values.  The  underscript /refers  to 

function  values  while  g refers  to  gradient  values.  RMSE  refers  to  the  original  nine  points 
while  Re  refers  to  the  analytical  RMSE  calculated  over  the  entire  design  domain 


bo 

bi 

b2 

b3 

b4 

b5 

RMSEf 

RMSEg 

Ref 

Re„ 

l2 

y MmBias 

0.717 

1.933 

1.933 

2.190 

2.200 

2.190 

1.294 

4.111 

0.374 

1.574 

/\ 

yf 

0.556 

2.667 

2.667 

2.667 

3.000 

2.667 

0.588 

3.203 

0.796 

1.515 

0.623 

1.913 

1.913 

2.187 

2.300 

2.187 

1.323 

4.094 

0.388 

1.573 

/V 

V, 

-0.675 

3.266 

3.266 

3.589 

3.750 

3.589 

1.206 

2.479 

1.466 

2.214 

The  largest  gains  in  accuracy  were  noted  in  the  function  approximation,  while  the 
gradient  approximation  shown  in  Table  20  did  not  gain  from  using  gradient  data;  if 
anything  it  got  slightly  worse.  This  unexpected  result  might  be  explained  by  Figure  27  b) 
and  d).  The  true  gradients  are  cubic  polynomials  of  the  two  variables,  while  the  gradients 
of  the  two  approximations  are  only  linear.  When  significant  curvature  is  present,  a linear 
function  cannot  give  good  approximation  over  large  intervals.  In  addition  using  identity 
matrix  as  a weighting  matrix  gave  the  most  inaccurate  results.  Moreover,  the 
approximation  obtained  from  the  iterative  covariance  algorithm  is  almost  identical  to 
y MmBias  ’ the  quadratic  polynomial  that  has  the  minimum  analytical  RMSE  over  the  entire 
design  domain.  This  last  approximation  is  obtained  by  finding  the  parameters  bj  of  a 
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quadratic  polynomial  of  Eq.  (6.42)  that  minimize  the  analytical  expression  of  the  RMSE, 
Re  over  the  entire  domain 


^{True Function  - yMinBias ) dV 

Re  = f ? (6.46) 

jdV 


Increasing  the  order  of  the  approximation 

Next,  the  effect  of  using  derivative  data  to  increase  the  order  of  the  approximation 

was  considered.  If  the  number  of  data  points  is  enough  to  obtain  a quadratic  polynomial 

approximation  from  function  values  only,  then  the  number  of  points  n satisfies 


n > nb„=2  > 


( ndv  + 1 )(ndv  + 2) 


(6.47) 


where  nb.p=2  is  the  number  of  parameters  in  a quadratic  polynomial  and  njv  is  the  number 
of  variables  of  the  quadratic  polynomial.  At  each  point  there  are  n^  derivatives,  so  that 
the  total  number  of  data  available  from  n points  is 

n,  > -n‘h  - l)^h  - 2)  {ndv  + 1)  (6.48) 


The  number  coefficients  of  a cubic  polynomial  can  be  obtained  from  Eq.  (6.48)  as 


„ ^(«<iv+i)K,  + 2K+3) 

nb,P= 3 — c 


(6.49) 


The  number  of  coefficients  nc  to  be  estimated  in  the  covariance  matrix  C of  Eq.  (6.38)  by 
the  iterative  procedure  are 


(ndv  + \)(ndv  + 2) 


(6.50) 


It  follows  that  for  n^v  > 1 (i.e.,  for  problems  with  more  than  one  design  variable) 
there  is  enough  information  to  fit  a cubic  polynomial  using  function  and  gradient 
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information  when  there  are  enough  points  to  fit  a quadratic  function  using  only  function 
values,  since 

^ (Ach  + + 2)  (n  +1x  > ("rfv  + + 2)(wav  + 3)  , (/7a-  + !)(”*  + ~ ^ (6  51) 

' 2 K dv  ’ 6 2 

The  following  examples  present  the  results  obtained  by  creating  cubic  polynomial 

approximations  using  function  and  gradient  information  at  nine  points,  that  constitute  a 

factorial  design  in  two  design  variables  in  the  range  +1.  The  nine  points,  the  true  function 

as  well  as  the  quadratic  approximation  based  on  function  values  only,  are  the  same  as  in 

Example  1. 

th 

Example  2:  Cubic  approximation  of  4 order  polynomial 

The  case  was  considered  in  which  the  true  function  is  a full  quartic  polynomial,  as 

in  Eq.  (6.43),  and  the  two  approximations  being  constructed  are: 

• y , cubic  approximation  obtained  using  the  function  values  and  the 

O 

gradients  at  nine  points. 

• y f quadratic  approximation  obtained  using  only  function  values  at  the 
same  nine  design  points  used  for  y . 

O 

The  covariance  and  weight  matrix  obtained  after  seven  iterations  of  the  iterative 
covariance  algorithm  with  a convergence  criteria  equal  to  1 O’4  were 


0. 1 233 

0 

0 

8.1 1 17 

0 

0 

0 

3.6674 

3.5098 

W = I9® 

0 

3.2413 

-3.1019 

(6.52) 

0 

3.5098 

3.6674 

0 

-3.1019 

3.2413 

It  is  interesting  to  note  that  the  correlation  matrix  shows  strong  correlation 
between  the  gradients,  and  none  between  the  gradients  and  the  function  values,  even 
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though  no  simplifications  were  imposed  on  the  form  of  the  correlation  matrix.  The 
quadratic  approximation  obtained  using  only  function  values  y f , is  obviously  the  same 

one  presented  earlier  in  Table  20,  and  it  is  reported  again  for  completeness  in  Table  21 . 


Table  21 : Parameters  and  error  measures  ol  the  cubic  approximation  yg , obtained  from 


function  and  gradient  values  and  the  quadratic  approximation  yf  obtained  using  only 

function  values.  The  underscript/ refers  to  function  values  and  g refers  to  gradient  values. 
RMSE  refers  to  the  original  nine  points  and  Re  to  the  analytical  RMSE^ 


Bo 

b i 

b2 

b3 

b4 

bs 

be 

67 

bs 

bs 

/V 

y.f 

0.556 

2.667 

2.667 

2.667 

3.000 

2.667 

- 

- 

- 

- 

RMSEef 

0.588 

RMS  Eg 
3.203  * 

Ref 

0.793 

Reg 
1 .685 

bo 

bi 

b2 

b3 

b4 

65 

be 

by 

bs 

b9 

A 

yK 

0.023 

1.00 

1.00 

3.066 

2.918 

3.066 

1.0 

1.0 

1.0 

1.0 

RMSEef 

0.351 

RMS  Eg 
1.913 

Ref 

0.488 

Reg 

1.089 

Table  21  shows  that  the  cubic  approximation  based  on  gradient  information  is 
more  accurate  than  the  approximation  derived  from  function  values  only.  Moreover 
comparing  the  results  shown  in  Table  21  with  the  results  shown  in  Table  20  it  appears 
that  using  the  gradient  information  to  increase  the  order  of  the  approximation  increases 
the  error  in  the  function  predictions,  but  it  reduced  by  50%  the  errors  in  the  predictions  of 
the  gradients. 


Example  3:  Ten  design  variable  case 

An  additional  numerical  experiment  was  carried  out  using  ten  design  variables. 
The  true  function  was  a full  quartic  polynomial  in  the  ten  design  variables  with  all 
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coefficients  equal  to  1.0.  Five  different  approximations  were  constructed  based  on  the 
values  of  70  design  points.  The  points  were  chosen  using  by  LHS  method: 

• yK  onau  quadratic  approximation  obtained  using  function  values  and 

gradients  at  70  design  points. 

• yK  cubu  cubic  approximation  obtained  using  function  values  and  gradients 
at  70  design  points. 

• y,  quadratic  approximation  obtained  using  only  function  values  at  the 
same  70  design  points. 

• >7  Quad  quadratic  approximation  obtained  using  function  values  and 

gradients  at  70  design  points  and  the  Identity  matrix  as  a weighting  matrix. 

• y ! CuhjL.  cubic  approximation  obtained  using  function  values  and  gradients 

at  70  design  points  and  the  Identity  matrix  as  a weighting  matrix. 

Note  that  a quadratic  polynomial  in  10  design  variables  has  66  coefficients  and 
therefore  the  approximation  based  on  function  values  only,  y f will  have  just  about 

enough  information  to  obtain  the  66  parameters.  The  results  obtained  are  shown  in  Table 
22,  where  due  to  their  large  number,  the  coefficients  of  each  approximation  were  omitted. 
In  Table  22  the  quadratic  and  cubic  polynomials  having  the  minimum  possible  bias  error 
are  also  presented.  Note  that  these  two  last  approximations  are  guaranteed  to  have  the 
lowest  analytical  Re f for  the  their  class  of  polynomials,  however  they  are  not  guaranteed 
to  have  the  lowest  sum  of  the  analytical  RMSE  of  the  gradients,  Re%> sum.  As  can  be  seen, 
the  iterative  covariance  algorithm  gives  a much  more  accurate  approximations  than  the 
approximation  based  on  function  values  only.  However  in  contrast  with  the  two  design 
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variables  case,  the  approximation  obtained  using  the  identity  matrix  as  a weighting 
matrix  gives  the  best  approximation.  In  order  to  establish  if  the  identity  matrix 
consistently  gives  better  results  for  this  higher  dimensional  problem,  the  same  problem 
was  repeated  using  100  and  150  design  points.  For  these  two  last  cases  the  iterative 
covariance  algorithm  gave  slightly  better  results  than  the  approximations  based  on  the 
identity  matrix  as  a weighting  matrix,  suggesting  that  as  the  number  of  data  available 
increases  the  iterative  covariance  matrix  algorithm  becomes  more  and  more  accurate. 


Table  22:  Response  surfaces  obtained  from  the  true  function  and  true  gradient  values  at 
70  design  points.  The  true  function  is  a quartic  polynomial  in  ten  design  variables.  Re 
indicates  the  analytical  RMSE  calculated  over  the  whole  design  domain.  The  underscript/ 
refers  to  function  values  and  g refers  to  gradient  values.  The  Reg  sUm  is  the  sum  of  the  ten 
Re  of  the  gradients.  Min  Bias  refers  to  quadratic  and  cubic  polynomials  that  minimize  the 
analytical  bias  error  of  the  function. 


Ref 

R^Sum 

/V 

y g,Quad 

5.806 

55.58 

y g,Cub 

4.246 

46.18 

y r 

21.62 

/\ 

y I , Quad 

4.532 

46.21 

/\ 

y I , Cubic 

4.152 

45.80 

Min  Bias  Quad 

3.819 

50.01 

Min  Bias  Cubic 

2.692 

39.05 

Noise  Errors 


In  this  section  the  primary  focus  of  the  investigation  is  on  noise  error.  Normally 
distributed  noise  is  added  to  the  values  of  the  true  function  and  then  an  approximation  is 
fitted  using  the  noisy  data.  The  true  function  is  set  to  be  a quadratic  polynomial  as 


f (X| , Xj  ) — 1 + X|  + X-,  + X-,  + X|X2  + X- 


(6.53) 
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Random  errors  from  a normal  distribution  are  added  to  the  value  of  the  true  function  to 
generate  noisy  data  that  are  subsequently  used  to  obtain  the  response  surface  expressions. 
Random  error  is  added  in  the  following  fashion 

^(x,)  = /(x,)  + f,,  x,  = [x„,x2(]  and  s,  =N( 0,cr)  (6.54) 

c^Oq)  = + ^ , x_  = ^ j for  j = j 2 and  s,  = N( 0, a2 ) (6.55) 

dx,  dx. 

The  two  approximations  generated  from  the  data  are: 

• y approximation  obtained  using  the  function  and  gradients  values  at  nine 

o 

design  points.  The  covariance  matrix  used  in  the  iterative  covariance 
algorithm  is  the  fully  correlated  covariance  matrix  of  Eq.  (6.38). 

• yf  approximation  obtained  using  only  function  values  at  the  same  nine 

design  points  used  for  y . 

O 

Due  to  the  random  nature  of  the  error,  in  order  to  make  a reasonable  comparison 
between  the  two  approximations,  different  sets  of  different  randomly  generated  errors 
were  used.  The  same  sets  were  used  to  obtain  y and  yf . 

Example  1 : Approximation  of  2 order  polynomial  with  random  errors  cr"  = 0.03 

In  this  first  example  noise  with  variance  cr  = 0.03  is  added  to  the  true  function 

and  to  its  derivatives.  Ten  sets  of  random  errors  were  generated  and  added  to  the  values 
of  the  true  function,  y f . The  results  obtained  are  presented  in  Table  23  and  Table  24.  In 

these  tables,  the  results  indicated  by  Re/mi  refer  to  the  root  error  mean  square  of  the 

approximating  function  calculated  at  1 01  points  while  Regioi  refers  to  the  root  mean 

2 

square  error  of  the  gradients  at  the  same  101"  points. 
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Table  23:  Response  surfaces  obtained  from  the  true  function  and  true  gradient  values 
with  added  noise.  The  noise  added  had  variance  a1  = 0.03  for  both  function  and  gradient 
values.  Errors  measures  Re  are  calculated  at  101  check  points.  The  underscript/ refers  to 
function  values  and  g refers  to  gradient  values.  The  number  1 or  2 refers  to  the  design 
variable  number.  Av  means  average. 


Set 

b o 

b\ 

b2 

b~i 

b4 

b 5 

Ref,ioi 

Regi,ioi 

Reg2,ioi 

1 

1.0651 

1.0135 

1.0055 

0.9927 

0.9863 

0.9860 

0.0588 

0.0179 

0.0190 

2 

0.9193 

0.9752 

0.9470 

1.0515 

1.0031 

1.0030 

0.0726 

0.0650 

0.0531 

3 

1.0033 

0.9923 

0.9867 

1.0095 

1.0227 

0.9592 

0.0189 

0.0189 

0.0511 

4 

0.9966 

1.0350 

1.0270 

0.9929 

1.0375 

1.0619 

0.0376 

0.0421 

0.0801 

5 

1.0033 

0.9923 

0.9867 

1.0095 

1.0227 

0.9592 

0.0189 

0.0189 

0.0511 

6 

1.0587 

1.0928 

1.0728 

1.0292 

1.0593 

1 .0049 

0.1008 

0.1047 

0.0808 

7 

0.9739 

1.1044 

1.0681 

0.9938 

1.0205 

1.0088 

0.0773 

0.1053 

0.0699 

8 

1.0283 

0.9628 

0.9855 

1.0246 

0.9777 

0.9838 

0.0407 

0.0487 

0.0271 

9 

0.9003 

0.9709 

1.0318 

0.9921 

1.0081 

1.0481 

0.0908 

0.0308 

0.0647 

10 

Av. 

1.0555 

0.9141 

0.9974 

1.0657 

0.9957 

0.9440 

0.0816 

0.0598 

0.1151 

0.0567 

0.0654 

0.0562 

Table  24  Response  surfaces  obtained  from  the  true  function  values  with  added  noise.  The 
noise  had  variance  <j  = 0.03.  The  errors  measures  Re  are  calculated  at  101"  check  points. 
The  underscript  / refers  to  function  values  and  g refers  to  gradient  values.  The  number  1 
or  2 refers  to  the  design  variable  number.  Av  means  average. 


Set 

bo 

b i 

b 2 

&3 

b\ 

b 5 

Rer,  i o i 

Reofioi 

Reg2,ioi 

1 

0.9304 

1.0155 

1.0229 

1.1265 

1.0548 

1 .0467 

0.0490 

0.1517 

0.0671 

2 

1.1521 

0.9986 

0.9945 

0.7644 

1.0428 

0.9449 

0.0921 

0.2759 

0.0692 

3 

1.0458 

0.9323 

1.0761 

0.9278 

0.8780 

0.9396 

0.0779 

0.1293 

0.1258 

4 

1.1633 

1.0106 

1.0230 

0.9160 

0.8722 

0.9353 

0.1260 

0.1236 

0.1085 

5 

1.0458 

0.9323 

1.0761 

0.9278 

0.8780 

0.9396 

0.0779 

0.1293 

0.1258 

6 

0.9677 

1 .0947 

0.9920 

1.1646 

0.8514 

1 .0044 

0.0936 

0.2309 

0.0872 

7 

0.8701 

0.9703 

1.0123 

0.9767 

0.9470 

1.2008 

0.0964 

0.0508 

0.2366 

8 

1.2869 

1.1534 

0.9546 

0.8953 

1.0263 

0.7529 

0.2083 

0.1966 

0.2922 

9 

0.8999 

0.9428 

1.0425 

1.0323 

1.0205 

1.0062 

0.0972 

0.0696 

0.0448 

10 

Av. 

1.1155 

0.9277 

1.0783 

1.0332 

0.9389 

0.8933 

0.1168 

0.1035 

0.0894 

0.1447 

0.1513 

0.1308 

Comparing  Table  23  and  Table  24,  it  appears  that  response  surface 
approximations  obtained  combining  function  and  gradient  information  are  more  accurate 
than  the  approximation  obtained  using  only  function  values.  In  particular,  using  gradient 
data  in  the  fitting  procedure  drastically  increases  the  accuracy  of  the  gradients 
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approximations.  Moreover  when  using  both  function  and  gradient  data,  despite  the 
amount  of  data  being  three  times  larger  than  the  case  of  approximation  obtained  from 
function  values  only,  the  average  Re/ioi  for  the  10  sets  of  random  errors  decreased  only 
by  50%. 

The  iterative  covariance  algorithm  was  used  in  all  the  cases  of  regressions  based 
on  function  and  gradient  information.  Convergence  was  assumed  when  none  of  the  b 
coefficients  changed  by  more  than  10.  The  most  accurate  covariance  matrix  estimated 
by  the  procedure  was  found  for  design  number  5 as 


C = 10'3 


31.309 

-11.958 

4.481 


-11.958 

32.522 

-2.040 


4.481 

-2.040 

26.360 


(6.49) 


The  covariance  matrix  of  this  case  is  estimated  very  well  considering  that  the 
noise  was  generated  from 


C = 10~3 


30.0 

0.0 

0.0 

0.0 

30.0 

0.0 

0.0 

-0.0 

30.0 

(6.56) 


The  worst  estimation  of  the  covariance  matrix  was  found  for  design  number  1 0 as 


C = 10'3 


18.743 

-5.125 

0.429 


-5.125 
1 1 .042 
-3.858 


0.429 

-3.858 

5.838 


(6.57) 


Note  that  for  cases  5 and  10  the  corresponding  approximation  are  also  best  and 


worst. 
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Example  2:  Approximation  of  2nd  order  polynomial  with  random  errors  g_  = 0.06 

A comparison  between  approximations  obtained  by  using  function  and  gradient 
data  and  function  data  only,  was  repeated  for  noise  generated  from  normal  distribution  of 
variance  cT  = 0.06.  The  true  function  with  no  noise  was  still  given  by  Eq.  (6.46).  The 
results  obtained  for  this  new  case  are  presented  in  Table  25  and  Table  26.  The  benefit  ot 
using  the  gradient  values  in  obtaining  the  approximation  to  the  true  function  is  once  more 
evident.  Major  gains  in  accuracy  obtained  for  the  gradients  expressions  are  noticeable. 


Table  25:  Response  surfaces  obtained  from  the  true  function  and  gradient  values  with 
added  noise.  The  noise  had  variance  a2  = 0.06  for  both  function  and  gradient  values.  The 
errors  measures  Re  are  calculated  at  1 0 1 2 check  points.  The  underscript /refers  to 
function  values  and  g refers  to  gradient  values.  The  number  1 or  2 refers  to  the  design 

variable  number.  Av  means  average. 


Set 

bo 

b\ 

bi 

^3 

b\ 

b 5 

Ref.ioi 

Re°i,ioi 

Reg2,ioi 

1 

1 .0329 

1.0270 

1.0477 

0.9807 

1 .05 1 6 

0.9888 

0.0434 

0.0462 

0.0579 

2 

0.9241 

0.9949 

0.9871 

0.9649 

0.9409 

0.9762 

0.0992 

0.0537 

0.0461 

3 

1.0582 

0.9412 

0.9240 

0.9895 

1.2206 

1.0209 

0.1124 

0.1420 

0.1514 

4 

0.8701 

0.9802 

1.0031 

1.0178 

0.9028 

1.0077 

0.1264 

0.0635 

0.0574 

5 

Av 

0.8647 

0.8756 

0.9547 

1.0087 

0.9505 

0.9872 

0.1579 

0.1079 

0.1281 

0.0867 

0.0557 

0.0737 

Table  26:  Response  surfaces  obtained  from  the  true  function  values  with  added  noise. 
The  noise  had  variance  a2  = 0.06.  The  error  measures  Re  are  calculated  at  10E  check 
points.  The  underscript / refers  to  function  values  and  g refers  to  gradient  values.  The 
number  1 or  2 refers  to  the  design  variable  number.  Av  means  average. 


Set 

bo 

bx 

b 2 

bi 

b 4 

b 5 

Ref.ioi 

Regi.ioi 

Reg2,ioi 

1 

0.7464 

1.0379 

1.0411 

1.2617 

0.7297 

1.1423 

0.1767 

0.3456 

0.2325 

2 

1.0533 

0.8266 

0.9747 

0.8802 

0.9809 

0.9108 

0.1134 

0.2229 

0.1076 

3 

0.7905 

1.0504 

0.9806 

1.3331 

1.1442 

1.0789 

0.1380 

0.4007 

0.1261 

4 

0.8462 

0.9827 

1.0161 

0.9221 

0.8906 

1.1422 

0.1464 

0.1123 

0.1784 

5 

Av. 

0.7632 

0.8037 

0.9442 

1.0332 

0.9970 

1.0780 

0.2332 

0.1615 

0.2001 

0.2563 

0.1067 

0.1502 
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Example  3:  Approximation  of  2 order  polynomial  with  random  errors  a = 0.03  on  the 
function  values  and  cr~  = 0.06  on  the  gradient  values 

Next  the  case  of  added  noise  with  two  different  variances,  one  for  the  function 
and  one  for  the  gradients  data,  was  considered.  The  random  noise  added  to  the  true 
function  values  had  a variance  cf  = 0.03,  while  the  noise  added  to  the  true  value  ol  the 

gradients  had  a variance  of  cr  = 0.06. 

The  results  obtained  from  the  fit  that  used  function  and  gradient  information  are 
presented  in  Table  27,  while  the  results  obtained  for  the  fit  obtained  using  only  function 
values  are  shown  in  Table  28.  Comparison  of  Table  27  and  Table  28  suggests  that  there  is 
still  an  advantage  in  including  gradient  information  in  constructing  an  approximation 
even  when  the  noise  in  the  functions  and  the  gradients  is  large. 


Table  27:  Response  surfaces  obtained  using  function  and  gradient  having  different  noise 
levels.  The  noise  added  to  function  values  had  variance  cf  = 0.03  while  the  noise  added 
to  the  gradient  values  had  variance  of  cr  = 0.06.  The  error  measures  Re  are  calculated  at 
101 2 check  points.  The  underscript/ refers  to  function  values  and  g refers  to  gradient 
values.  The  number  1 or  2 refers  to  the  design  variable  number.  Av  means  average 


Set 

bo 

b\ 

b2 

^3 

b<\ 

b 5 

73 

4? 

jo 

: 

i 

Regi;ioi 

Reg2,ioi 

1 

0.8647 

0.8756 

0.9547 

1.0087 

0.9505 

0.9872 

0.1579 

0.1281 

0.0557 

2 

1.2342 

1.1079 

0.9611 

0.7532 

0.9996 

0.8817 

0.1533 

0.3074 

0.1434 

3 

0.9443 

1 .0049 

1.0617 

0.9754 

0.9478 

1.0855 

0.0598 

0.0421 

0.1212 

4 

1.1304 

0.9948 

0.9044 

0.9470 

1.0046 

0.9137 

0.1047 

0.0620 

0.1389 

5 

Av. 

0.8984 

1.0365 

0.8968 

1.0640 

1.0391 

1.0390 

0.0960 

0.1143 

0.0862 

0.1251 

0.1151 

0.1148 
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Table  28:  Response  surfaces  obtained  using  only  function  values  with  added  noise  having 
variance  cr“  = 0.03.  The  error  measures  Re  are  calculated  at  101“  check  points.  The 
underscript  / refers  to  function  values  and  g refers  to  gradient  values.  The  number  1 or  2 

refers  to  the  design  variable  number.  Av  means  average. 


Set 

b o 

bi 

b2 

h 

b4 

b$ 

Ref,  101 

Regi,ioi 

Reg2.ioi 

1 

0.7632 

0.8037 

0.9442 

1.0332 

0.9970 

1.0780 

0.2332 

0.2001 

0.1067 

2 

1.3312 

0.9428 

0.9263 

0.7853 

1.0684 

0.7551 

0.2096 

0.2599 

0.2976 

3 

1.0562 

1.0651 

1.0807 

0.8518 

0.9753 

1.0221 

0.0773 

0.1853 

0.0859 

4 

1.0887 

1.0504 

0.9099 

0.9600 

1.1519 

0.9735 

0.1043 

0.1121 

0.1300 

5 

Av. 

0.9815 

1.0806 

0.9999 

0.9457 

1.0500 

1.0763 

0.0586 

0.1366 

0.1066 

0.1728 

0.0937 

0.1427 

i 2 

Example  4:  Approximation  of  2 order  polynomial  with  random  errors  q_  = 0.3 

The  case  of  added  noise  with  a large  variance  was  also  considered.  The  random 

noise  added  to  the  true  function  values  and  the  gradients  had  a variance  cr  = 0.3.  The 
results  obtained  from  the  fit  that  used  function  and  gradient  information  are  presented  in 
Table  29,  while  the  results  obtained  for  the  fit  obtained  using  only  function  values  are 
shown  in  Table  30.  Comparison  of  Table  29  and  Table  30  suggests  that  there  is  still  an 
advantage  in  including  gradient  information  in  constructing  an  approximation  even  when 
the  noise  in  the  gradients  is  large.  However  the  gains  in  accuracy  obtained  by  including 
the  gradient  information  progressively  reduce  as  the  noise  in  the  gradient  information 


increases. 
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Table  29:  Response  surfaces  obtained  using  function  and  gradient  having  the  same  noise 
levels.  The  noise  added  to  function  values  and  the  gradient  had  variance  cr  = 0.3.  The 
error  measures  Re  are  calculated  at  101"  check  points.  The  underscript / refers  to  function 
values  and  g refers  to  gradient  values.  The  number  1 or  2 refers  to  the  design  variable 

number.  Av  means  average 


Set 

bo 

b i 

&2 

h 

b4 

b$ 

Ref,  ioi 

RegUOt 

Reg2,ioi 

1 

0.6191 

0.7402 

0.9376 

1.0195 

0.9389 

1.1195 

0.3706 

0.2619 

04564 

2 

0.7782 

1.2619 

1.2247 

1.0345 

1.1003 

1.1433 

0.2641 

0.2692 

0.2798 

3 

0.7926 

0.9909 

0.9405 

1.1607 

0.8721 

0.9561 

0.1817 

0.2013 

0.1068 

4 

1.1437 

1.1736 

0.5865 

0.9638 

1.0744 

0.9543 

0.2862 

0.1822 

0.4172 

5 

Av 

1.0111 

0.9763 

0.9581 

1.0311 

1.0728 

0.8705 

0.0563 

0.2317 

0.0603 

0.1950 

0.1584 

0.2237 

Table  30:  Response  surfaces  obtained  using  only  function  values  with  added  noise  having 
variance  a =0.3.  The  error  measures  Re  are  calculated  at  101“  check  points.  The 
underscript/ refers  to  function  values  and  g refers  to  gradient  values.  The  number  1 or  2 
refers  to  the  design  variable  number.  Av  means  average. 


Set 

b o 

b\ 

bi 

h 

b4 

bs 

Ref,  ioi 

Regi,ioi 

Reg2,ioi 

1 

0.0398 

0.9450 

1.1996 

1.5460 

0.7064 

1.2874 

0.7196 

0.6616 

0.4260 

2 

1.1419 

1.0876 

1.0956 

0.7411 

0.4707 

0.8556 

0.2151 

0.4405 

0.3643 

3 

0.3195 

0.9992 

1 .0494 

1.0920 

0.8413 

1.7517 

0.4604 

0.1408 

0.8829 

4 

1.2292 

1.0840 

0.4836 

0.9212 

1.0386 

0.7425 

0.3363 

0.1265 

0.5978 

5 

Av 

1.1450 

0.7860 

1.2407 

0.7718 

0.6142 

0.8090 

mt»Wmm>OOW>HW<ONMIIOWMW>WDO«tD 

0.2463 

0.3955 

IIHlimiWIWW<tMWt<Ct>eMM»IIIMIIim 

0.4090 

0.3556 

WOOMPMI—CMfllltMIBMBOmtHMHIO— <010— ( 

0.3977 

0.5337 

Example  5:  Approximation  of  2nd  order  polynomial  with  correlated  random  errors 
Finally  the  case  of  correlated  errors  was  considered.  The  correlation  used  to 

generate  the  noise  to  be  added  to  the  function  and  gradient  values  at  each  design  point 
was  such  that  the  correlation  between  the  function  values  and  the  first  gradient  was  equal 
to  0.3,  the  correlation  between  the  function  and  the  second  gradient  was  equal  to  0.5  and 
the  correlation  between  the  gradients  was  equal  to  0.4.  Ten  sets  of  random  errors  with 
variances  equal  to  0.05,  0.005  and  0.1  for  the  function,  first  gradient  and  second  gradient, 
respectively  were  generated.  The  results  obtained  are  presented  in  Table  31,  where  in  this 
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case  Re  indicates  the  analytical  root  mean  square  error.  As  can  be  seen,  using  the  iterative 
covariance  matrix  algorithm  usually  gave  the  most  accurate  results.  In  particular  the 
results  obtained  from  using  only  function  values  were  always  much  less  accurate  than  the 
results  obtained  using  the  iterative  covariance  algorithm.  Using  the  gradient  data  was 
useful  in  terms  of  accuracy  even  when  no  weighting  was  used,  that  is  the  weighting 
matrix  was  equal  to  the  identity  matrix. 


Table  31:  Accuracy  of  the  response  surfaces  obtained  from  the  true  function  and  true 
gradient  values  with  correlated  added  noise.  Analytical  RMSE  of  the  approximation  based 
only  on  function  values,  yf , of  the  approximation  based  on  function  values  and  gradients 

using  the  iterative  covariance  algorithm  y , and  of  the  approximation  based  on  function 

O 


values  and  gradients  using  the  identity  matrix  as  a weighting  matrix,  y , . 


Set 

/\ 

yg 

/\ 

y f 

A 

y, 

1 

0.105 

0.320 

0.153 

2 

0.128 

0.196 

0.173 

3 

0.182 

0.199 

0.181 

4 

0.045 

0.153 

0.103 

5 

0.130 

0.204 

0.189 

6 

0.068 

0.097 

0.066 

7 

0.021 

0.114 

0.053 

8 

0.054 

0.125 

0.078 

9 

0.032 

0.105 

0.070 

10 

0.026 

0.203 

0.067 

Av. 

0.079 

0.172 

0.113 

Reduction  of  terms  in  the  approximations 

The  case  of  a true  function  that  is  described  by  a quadratic  polynomial  in  two 

design  variables,  x / and  X2,  as  in  Eq.  (6.46)  was  considered  and  normally  distributed 
random  errors  as  in  Eq.  (6.47)  and  Eq.  (6.48)  are  added  to  the  values  of  the  true  function 
at  the  design  points.  The  approximation  created  are 
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• y cubic  approximation  obtained  using  the  function  values  and  the 

O 

gradients  at  nine  design  points. 

• yf  quadratic  approximation  obtained  using  only  function  values  at  the 
same  nine  design  points  used  for  y . 

o 

The  cubic  polynomial  approximations  obtained  based  on  function  and  gradients 
values  for  this  case  had  coefficients  very  similar  to  the  coefficients  shown  in  Table  23  for 
the  quadratic  part  and  coefficients  of  the  order  of  0.005  for  cubic  terms. 

After  approximating  the  3 order  polynomial  on  the  data  set,  it  was  found  that  the 
tsiat  as  defined  by  Eq.  (6.30)  for  third  order  terms  were  all  in  the  range  0.05  - 0.7.  An 
elimination  procedure  based  on  discarding  the  term  with  the  lowest  tsta,  was  initiated  and 
was  continued  until  all  remaining  terms  all  had  tstat  higher  than  2.0.  The  procedure  was 
applied  to  all  data  sets  shown  in  Table  23  and  always  ended  by  eliminating  all  cubic 
terms.  Therefore,  the  results  obtained  are  equivalent  to  those  one  presented  in  Table  23. 

Conclusion  on  including  gradient  information  in  constructing  approximations 

Using  simple  mathematical  examples,  it  has  been  shown  that  the  inclusion  of 

gradient  information  can  be  very  useful  in  improving  the  quality  of  the  approximations. 
Some  cases  have  been  met  where  using  the  identity  matrix  as  a weighting  matrix 
produced  slightly  better  results  than  using  the  proposed  algorithm.  However  in  many 
more  cases  the  iterative  covariance  algorithm  obtained  much  more  accurate 
approximations  than  any  other  method  considered.  In  particular,  the  gradients  of  the 
response  surface  approximations  built  using  function  and  gradient  information  were  more 
accurate  than  the  gradient  of  response  surfaces  that  were  constructed  using  only  function 
values.  On  the  other  hand  based  on  the  much  larger  amount  of  data  that  is  available  when 
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gradients  are  considered  the  accuracy  in  the  function  values  obtained  from  the 
approximations  achieved  was  less  than  expected. 

Further  investigations  are  required  in  the  field  of  approximations  based  on 
gradient  and  function  information.  A generalization  of  the  methods  found  in  analysis  of 
modeling  multiple  responses  as  in  Khuri  and  Cornell  (1996)  and  in  Robinson  (2000)  is 
needed. 


CHAPTER  7 

CONCLUDING  REMARKS 

The  use  of  response  surface  approximations  in  expensive  structural  optimization 
problems  was  investigated  in  this  study.  The  use  of  response  surface  approximations  in 
practical  problems  is  that  the  cost  of  constructing  the  approximations  is  subject  to  curse 
of  dimensionality.  The  curse  of  dimensionality  means  that  the  number  of  data  points 
required  to  construct  a response  surface  approximation  increases  fast  with  the  number  of 
design  variables.  Response  surface  approximations  are  thus  limited  to  practical 
application  that  have  only  few  design  variables. 

In  this  study,  methods  that  extend  the  applicability  of  response  surface  techniques 
to  problems  of  higher  dimensions  were  explored.  Two  methods  were  proposed: 
correction  response  surfaces  and  response  surfaces  based  on  gradient  values. 

Correction  response  surface  techniques  were  applied  successfully  to  two  panel 
structural  design  examples.  One  example  was  the  optimization  of  composite  hat-stiffener 
panel  subjected  to  stress  and  buckling  constraints  for  a blended-wing-body  transport. 
Coupling  a single  term  correction  response  surface  with  low-fidelity  methods  in  the  form 
of  closed  form  solutions  reduced  the  computational  effort  needed  to  obtain  an  optimal 
design.  A second  example  was  the  optimization  problem  of  a composite  bladed  stiffener 
wing  panel  for  a high  speed  civil  transport.  Linear  correction  response  surfaces  for  the 
ratio  and  the  difference  between  low-fidelity  and  high-fidelity  methods  of  analysis  were 
used.  It  was  shown  that  for  a given  amount  of  computational  resources,  correction 
response  surfaces  were  more  accurate  than  traditional  response  surfaces. 
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Gradients  are  often  cheaper  to  calculate  than  functions  in  structural  optimization 
problems.  The  possibility  of  including  gradient  information  in  the  construction  of 
response  surface  approximations  to  structural  responses  was  investigated.  A weighted 
least  square  procedure  that  uses  gradients  and  function  information  with  different  levels 
of  noise  was  obtained.  Simple  numerical  models  were  used  to  investigate  when  the 
inclusion  of  gradient  information  in  the  construction  of  a response  surface  is  beneficial. 

Including  gradient  information  in  the  construction  of  the  response  surface 
approximation  resulted  in  large  gains  in  the  accuracy  of  the  function  approximation  when 
only  modeling  errors  were  considered.  Even  when  only  noise  errors  were  considered, 
response  surface  approximations  obtained  combining  function  and  gradient  information 
resulted  to  be  more  accurate  than  the  approximation  obtained  using  function  values  only. 
The  increased  accuracy  obtained  by  using  gradient  information  for  the  response  surface 
approximations  in  the  presence  of  noise  errors  diminished  as  the  magnitude  of  the  noise 
increased.  Moreover  by  using  function  and  gradients  values  it  was  possible  to  increase 
the  order  of  the  polynomial  approximations  obtainable  from  the  data.  The  higher  order 
polynomial  approximations  were  more  accurate  than  the  ones  obtained  by  using  lower 
order  polynomial  approximations  based  on  function  values  only. 
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